Skip to content

FileExistsError error when psyplot is imported from multiple parallel processes #52

@schlunma

Description

@schlunma

Hello!

First of all, thanks for this great package!!

Problem description

We (the ESMValTool community) found an issue when importing psyplot from different parallel processes in a test environment, which leads to a race condition where the package finds that it has no config file and tries to create that, but two processes try that at the same time, resulting in a FileExistsError: [Errno 17] File exists: '/Users/runner/.config/psyplot'. Note that we only found this behavior for OSX.

Code Sample, a copy-pastable example if possible

An error on our CI machine using the import below is given here,

from psyplot.config.logsetup import _get_home

but we also found the same problem when using

import psyplot.project as psy

Please note that due to the race condition, the error cannot be reproduced consistently.

Expected Output

No error and no race condition.

Full error message

Details
 tests/unit/test_platform.py:4: in <module>
    from psyplot.config.logsetup import _get_home
../../../miniconda3/envs/esmvaltool/lib/python3.9/site-packages/psyplot/__init__.py:30: in <module>
    from psyplot.config.rcsetup import rcParams
../../../miniconda3/envs/esmvaltool/lib/python3.9/site-packages/psyplot/config/__init__.py:42: in <module>
    config_path = psyplot_fname()
../../../miniconda3/envs/esmvaltool/lib/python3.9/site-packages/psyplot/config/rcsetup.py:978: in psyplot_fname
    configdir = get_configdir()
../../../miniconda3/envs/esmvaltool/lib/python3.9/site-packages/psyplot/config/rcsetup.py:1028: in get_configdir
    os.makedirs(p)
../../../miniconda3/envs/esmvaltool/lib/python3.9/os.py:225: in makedirs
    mkdir(name, mode)
E   FileExistsError: [Errno 17] File exists: '/Users/runner/.config/psyplot'

Output of psyplot -aV

Details
psyplot:
  requirements:
    matplotlib: 3.5.2
    numpy: 1.22.3
    pandas: 1.4.2
    python: 3.8.13 | packaged by conda-forge
    xarray: 2022.3.0
  version: 1.4.2

NOTE

This is a bug report.

For requesting new features, use this template.

For changing existing features, use this template.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions