Skip to content

Commit a98f2eb

Browse files
GeigerJ2jan-janssensuperstar54
authored
Add AiiDA infrastructure (#11)
* Add universal_simple_to_aiida * Add universal_qe_to_aiida * Add CI setup to include AiiDA * Changes to quantum_espresso_workflow.py - Add `scale_structure` task - Prepend return dict keys of `generate_structures` with `s_` - Add function to serialize return atoms dict to pure python data types - Rename `name` -> `element` for `get_bulk_structure`, as `name` clashes with WG * Update .github/workflows/aiida.yml * Update quantum_espresso_workflow.py * Fix `aiida_to_jobflow_simple` * Fix `aiida_to_pyiron_base_simple` * Add gitignore * Make AiiDA WG construction for QE WF a helper function. * Add aiida_to_jobflow_qe.ipynb. Still some KeyError occurring. * Introduce `workflow_json_filename` variable * Add `jobflow_to_aiida_simple.ipynb` * Set WG version to latest commit on main. * Add `verdi presto` profile setup to `aiida.yml` * Fix aiida.yml * Add AiiDA profile creation to pyiron and jobflow CI YAML. * Update jobflow.yml * Update pyiron.yml * Fix `nodes` in JSON file created by AiiDA QE WF. Next debug jobflow run. * Found bug in WorkGraph. Continue now with task order. * Tasks are now in same order. Just need to add links. * Manual AiiDA WG creation now seems fixed. * Current WG working version * Adding python files used for being able to enter debugger. Remove once everything works. * Add intermediate JSON files of universal workflow reprs for debugging. * Bump WorkGraph to latest and fix a couple of issues - Bump aiida-workgraph to 0.5.0a6 - pickle_node has default `result` output socket, and will be set to None when exporting to json - for the get_volumes and get_energies tasks, we add input sockets based on the index of strain * All AiiDA notebooks should be there. * Set aiida-workgraph version in `environment.yaml` * Fix load_profile * Fix typos in CI YAML files * Cleaan up quantum_espresso_workflow.py * Add upterm to connect to Runner to debug AiiDA * Update .github/workflows/aiida.yml * Update .github/workflows/aiida.yml * Fix load_profile and remove upterm * Add pyiron_base_to_aiida_simple.ipynb * Start cleaning up repo. * Only need to fix pyiron_base_to_aiida_qe.ipynb * Adapt `workflow_json_filename`s * Latest file versions. Should we use nbstripout? * Fix universal_qe_to_aiida.ipynb * Comment out last cell of pyiron_base_to_aiida_qe.ipynb * Uncomment last cell so that CI doesn't seem successfull even though the WG run failed. * Update pyiron_base.py * Update pyiron_base.py --------- Co-authored-by: Jan Janssen <jan-janssen@users.noreply.github.com> Co-authored-by: superstar54 <xingwang1991@gmail.com>
1 parent b1b6117 commit a98f2eb

23 files changed

+7334
-512
lines changed

‎.github/workflows/aiida.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: aiida
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
pull_request:
7+
8+
jobs:
9+
build:
10+
11+
runs-on: ubuntu-latest
12+
13+
steps:
14+
- uses: actions/checkout@v4
15+
- uses: conda-incubator/setup-miniconda@v3
16+
with:
17+
auto-update-conda: true
18+
python-version: "3.12"
19+
environment-file: environment.yml
20+
auto-activate-base: false
21+
- name: Installation and setup
22+
shell: bash -l {0}
23+
run: |
24+
pip install -e adis_tools
25+
pip install -e python_workflow_definition
26+
conda install -c conda-forge jupyter papermill
27+
verdi presto --profile-name adis
28+
- name: Tests
29+
shell: bash -l {0}
30+
run: |
31+
export ESPRESSO_PSEUDO=$(pwd)/espresso/pseudo
32+
33+
papermill universal_simple_to_aiida.ipynb universal_simple_to_aiida_out.ipynb -k "python3"
34+
papermill universal_qe_to_aiida.ipynb universal_qe_to_aiida_out.ipynb -k "python3"
35+
36+
papermill aiida_to_pyiron_base_simple.ipynb aiida_to_pyiron_base_simple_out.ipynb -k "python3"
37+
papermill aiida_to_pyiron_base_qe.ipynb aiida_to_pyiron_base_qe_out.ipynb -k "python3"
38+
39+
papermill aiida_to_jobflow_simple.ipynb aiida_to_jobflow_simple_out.ipynb -k "python3"
40+
papermill aiida_to_jobflow_qe.ipynb aiida_to_jobflow_qe_out.ipynb -k "python3"

‎.github/workflows/jobflow.yml‎

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,14 +18,23 @@ jobs:
1818
python-version: "3.12"
1919
environment-file: environment.yml
2020
auto-activate-base: false
21-
- name: Tests
21+
- name: Installation and setup
2222
shell: bash -l {0}
2323
run: |
2424
pip install -e adis_tools
2525
pip install -e python_workflow_definition
2626
conda install -c conda-forge jupyter papermill
27+
verdi presto --profile-name adis
28+
- name: Tests
29+
shell: bash -l {0}
30+
run: |
2731
export ESPRESSO_PSEUDO=$(pwd)/espresso/pseudo
32+
2833
papermill universal_simple_to_jobflow.ipynb universal_simple_to_jobflow_out.ipynb -k "python3"
29-
papermill jobflow_to_pyiron_base_simple.ipynb jobflow_to_pyiron_base_simple_out.ipynb -k "python3"
3034
papermill universal_qe_to_jobflow.ipynb universal_qe_to_jobflow_out.ipynb -k "python3"
31-
papermill jobflow_to_pyiron_base_qe.ipynb jobflow_to_pyiron_base_qe_out.ipynb -k "python3"
35+
36+
papermill jobflow_to_pyiron_base_simple.ipynb jobflow_to_pyiron_base_simple_out.ipynb -k "python3"
37+
papermill jobflow_to_pyiron_base_qe.ipynb jobflow_to_pyiron_base_qe_out.ipynb -k "python3"
38+
39+
papermill jobflow_to_aiida_simple.ipynb jobflow_to_aiida_simple_out.ipynb -k "python3"
40+
papermill jobflow_to_aiida_qe.ipynb jobflow_to_aiida_qe_out.ipynb -k "python3"

‎.github/workflows/pyiron.yml‎

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,14 +18,23 @@ jobs:
1818
python-version: "3.12"
1919
environment-file: environment.yml
2020
auto-activate-base: false
21-
- name: Tests
21+
- name: Installation and setup
2222
shell: bash -l {0}
2323
run: |
2424
pip install -e adis_tools
2525
pip install -e python_workflow_definition
2626
conda install -c conda-forge jupyter papermill
27+
verdi presto --profile-name adis
28+
- name: Tests
29+
shell: bash -l {0}
30+
run: |
2731
export ESPRESSO_PSEUDO=$(pwd)/espresso/pseudo
32+
2833
papermill universal_simple_to_pyiron_base.ipynb universal_simple_to_pyiron_base_out.ipynb -k "python3"
29-
papermill pyiron_base_to_jobflow_simple.ipynb pyiron_base_to_jobflow_simple_out.ipynb -k "python3"
3034
papermill universal_qe_to_pyiron_base.ipynb universal_qe_to_pyiron_base_out.ipynb -k "python3"
31-
papermill pyiron_base_to_jobflow_qe.ipynb pyiron_base_to_jobflow_qe_out.ipynb -k "python3"
35+
36+
papermill pyiron_base_to_jobflow_simple.ipynb pyiron_base_to_jobflow_simple_out.ipynb -k "python3"
37+
papermill pyiron_base_to_jobflow_qe.ipynb pyiron_base_to_jobflow_qe_out.ipynb -k "python3"
38+
39+
papermill pyiron_base_to_aiida_simple.ipynb pyiron_base_to_aiida_simple_out.ipynb -k "python3"
40+
papermill pyiron_base_to_aiida_qe.ipynb pyiron_base_to_aiida_qe_out.ipynb -k "python3"

‎.gitignore‎

Lines changed: 202 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,202 @@
1+
# Byte-compiled / optimized / DLL files
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
6+
# C extensions
7+
*.so
8+
9+
# Distribution / packaging
10+
.Python
11+
build/
12+
develop-eggs/
13+
dist/
14+
downloads/
15+
eggs/
16+
.eggs/
17+
lib/
18+
lib64/
19+
parts/
20+
sdist/
21+
var/
22+
wheels/
23+
share/python-wheels/
24+
*.egg-info/
25+
.installed.cfg
26+
*.egg
27+
MANIFEST
28+
29+
# PyInstaller
30+
# Usually these files are written by a python script from a template
31+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
32+
*.manifest
33+
*.spec
34+
35+
# Installer logs
36+
pip-log.txt
37+
pip-delete-this-directory.txt
38+
39+
# Unit test / coverage reports
40+
htmlcov/
41+
.tox/
42+
.nox/
43+
.coverage
44+
.coverage.*
45+
.cache
46+
nosetests.xml
47+
coverage.xml
48+
*.cover
49+
*.py,cover
50+
.hypothesis/
51+
.pytest_cache/
52+
cover/
53+
54+
# Translations
55+
*.mo
56+
*.pot
57+
58+
# Django stuff:
59+
*.log
60+
local_settings.py
61+
db.sqlite3
62+
db.sqlite3-journal
63+
64+
# Flask stuff:
65+
instance/
66+
.webassets-cache
67+
68+
# Scrapy stuff:
69+
.scrapy
70+
71+
# Sphinx documentation
72+
docs/_build/
73+
74+
# PyBuilder
75+
.pybuilder/
76+
target/
77+
78+
# Jupyter Notebook
79+
.ipynb_checkpoints
80+
81+
# IPython
82+
profile_default/
83+
ipython_config.py
84+
85+
# pyenv
86+
# For a library or package, you might want to ignore these files since the code is
87+
# intended to run in multiple environments; otherwise, check them in:
88+
# .python-version
89+
90+
# pipenv
91+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
92+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
93+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
94+
# install all needed dependencies.
95+
#Pipfile.lock
96+
97+
# UV
98+
# Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
99+
# This is especially recommended for binary packages to ensure reproducibility, and is more
100+
# commonly ignored for libraries.
101+
#uv.lock
102+
103+
# poetry
104+
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
105+
# This is especially recommended for binary packages to ensure reproducibility, and is more
106+
# commonly ignored for libraries.
107+
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
108+
#poetry.lock
109+
110+
# pdm
111+
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
112+
#pdm.lock
113+
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
114+
# in version control.
115+
# https://pdm.fming.dev/latest/usage/project/#working-with-version-control
116+
.pdm.toml
117+
.pdm-python
118+
.pdm-build/
119+
120+
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
121+
__pypackages__/
122+
123+
# Celery stuff
124+
celerybeat-schedule
125+
celerybeat.pid
126+
127+
# SageMath parsed files
128+
*.sage.py
129+
130+
# Environments
131+
.env
132+
.venv
133+
env/
134+
venv/
135+
ENV/
136+
env.bak/
137+
venv.bak/
138+
139+
# Spyder project settings
140+
.spyderproject
141+
.spyproject
142+
143+
# Rope project settings
144+
.ropeproject
145+
146+
# mkdocs documentation
147+
/site
148+
149+
# mypy
150+
.mypy_cache/
151+
.dmypy.json
152+
dmypy.json
153+
154+
# Pyre type checker
155+
.pyre/
156+
157+
# pytype static type analyzer
158+
.pytype/
159+
160+
# Cython debug symbols
161+
cython_debug/
162+
163+
# PyCharm
164+
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
165+
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
166+
# and can be added to the global gitignore or merged into this file. For a more nuclear
167+
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
168+
#.idea/
169+
170+
# Ruff stuff:
171+
.ruff_cache/
172+
173+
# PyPI configuration file
174+
.pypirc
175+
176+
# Files of this package
177+
.vscode/
178+
input_tmp.in
179+
pyiron.log
180+
pyiron_draw.png
181+
python_workflow_definition/src/python_workflow_definition/__pycache__/
182+
test/
183+
mini/
184+
evcurve.png
185+
strain_0/
186+
strain_1/
187+
strain_2/
188+
strain_3/
189+
strain_4/
190+
aiida_to_pyiron_base_qe.json
191+
pyiron_base_to_aiida_qe.json
192+
jobflow_to_pyiron_base_simple.json
193+
aiida_to_pyiron_base_simple.json
194+
jobflow_to_aiida_simple.json
195+
pyiron_base_to_jobflow_simple.json
196+
aiida_to_jobflow_simple.json
197+
jobflow_to_pyiron_base_qe.json
198+
jobflow_to_aiida_qe.json
199+
aiida_to_jobflow_qe.json
200+
pyiron_base_to_aiida_simple.json
201+
pyiron_base_to_jobflow_qe.json
202+

0 commit comments

Comments
 (0)