AgnostiqHQ/covalent: v0.220.0

[0.220.0] - 2023-04-28 Authors Alejandro Esquivel ae@alejandro.ltd Faiyaz Hasan faiyaz@agnostiq.ai Sankalp Sanand sankalp@agnostiq.ai Co-authored-by: kessler-frost ssanand@hawk.iit.edu Co-authored-by: pre-commit-ci[bot] 66853113+pre-commit-ci[bot]@users.noreply.github.com Venkat Bala 15014089+venkatBala@users.noreply.github.com Co-authored-by: Santosh kumar 29346072+santoshkumarradha@users.noreply.github.com dwelsch-esi 116022979+dwelsch-esi@users.noreply.github.com Ara Ghukasyan 38226926+araghukas@users.noreply.github.com Will Cunningham wjcunningham7@users.noreply.github.com Co-authored-by: Venkat Bala venkat@agnostiq.ai Co-authored-by: dwelsch-memverge david.welsch@memverge.com Co-authored-by: Will Cunningham wjcunningham7@gmail.com Co-authored-by: Casey Jao casey@agnostiq.ai Prasanna Venkatesh 54540812+Prasy12@users.noreply.github.com Co-authored-by: kamalesh.suresh kamalesh.suresh@psiog.com Co-authored-by: Amalan Jenicious F amalan.jenicious@psiog.com Co-authored-by: Aravind-psiog aravind.prabaharan@psiog.com Co-authored-by: ArunPsiog arun.mukesh@psiog.com Co-authored-by: santoshkumarradha santosh@agnostiq.ai RaviPsiog 111348352+RaviPsiog@users.noreply.github.com Co-authored-by: RaviPsiog ravieja.gurram@psiog.com Akalanka 8133713+boneyag@users.noreply.github.com Co-authored-by: Scott Wyman Neagle scott@agnostiq.ai Scott Wyman Neagle wymnea@protonmail.com Co-authored-by: Manjunath PV manjunath.poilath@psiog.com Co-authored-by: RaviPsiog raviteja.gurram@psiog.com Aravind 100823292+Aravind-psiog@users.noreply.github.com Co-authored-by: Prasy12 prasanna.venkatesh@psiog.com ### Added Tutorial for hybrid neural network using Covalent, AWSBatch, and Qiskit Runtime. Environment variable that users can set to specify the location where Covalent can find their defined executors. Task group id in Electrons. Reconstruct postprocessing method. Postprocessing as electrons. Postprocessing class in postprocessing.py module for all the different postprocessing helper methods and algorithms. disable_run option added to enable "saving-only" option on covalent server and not executing the workflow register_triggers, stop_triggers functions added to LocalDispatcher class triggers parameter to the lattice metadata BaseTrigger, DirTrigger, TimeTrigger classes added available to be assigned to any lattice enabling the triggers feature TriggerLoader class added enabling loading of any kind of triggers including user defined ones without requiring installation CLI options to start covalent server in triggers only, and no triggers mode is_pending option added during redispatch to resume execution of a previously "saved-only", i.e pending workflow API routes added for Triggers server Added feature to support cancelling workflow dispatches Updating/adding new tests to improve code coverage API endpoint for redispatching. Unit and functional tests for redispatching. TransportGraphOps class for diffing operations on transport graphs. Added make derived dispatch method. Apply electron updates method to _TransportGraph. Workflow re-dispatching functionality. Added dispatcher_addr argument to ct.get_result similar to ct.dispatch so that it doesn't always fallback to using the default configured address Front-end pending unit tests for the GUI. Adding support for PostgresQL DB backend Added check for COVALENT_DATABASE_URL, if exists connect sqlalchemy engine using that Adding COVALENT_DATABASE_USER and COVALENT_DATABASE_PASSWORD environment variables Adding COVALENT_DATABASE_HOSTNAME and COVALENT_DATABASE_PORT environment variables for easy configuration Added TaskRuntimeError exception for executor plugin implementations to signal to Covalent that a task raised an unhandled exception while running in the executor backend. Added environment variable for a remote database backend Added support for mysql and postgresql Support for transferring the contents of folders to and from S3 buckets using the file transfer module. RTD for User Interface Minor GUI fixes Ability to use terminal on the GUI. ### Changed Postprocessing logic. Sublattice logic. Sublattices are now treated as electrons. Once the transport graph has been built, the status get changed to DISPATCHING at which point it is executed as another workflow. Modified ct.get_result to allow for status only requests Enhanced the Dockerfile to include builds from various sources and a differentiation between SDK and Server builds Location of function to load result from the database now moved to load module in covalent_dispatcher/_db folde. Changed decode-uri-component package version on webapp yarn-lock file. Changed json5 package version on webapp yarn-lock file. Reverted nightly frequency back to once a day Updated requirements.txt to include pyscopg2 Refactored execution.py into loosely coupled modular pieces Updated directory like default environment variable paths to avoid creating redundant nested directories when self-hosting Docs for Covalent's Slurm plugin updated with explanation for optional srun parameters. Electron errors are segregated by type; task runtime errors are stored in stderr while the error attribute of a node is reserved for exceptions raised by Covalent itself. When tasks fail in a workflow, the Lattice ErrorCard in the UI summarizes the failed tasks. Width of lattice name column on dispatch list GUI. Optimzing larger graphs for better performance. Running migrations automatically if none have run in the past (fresh installs, after purging) Single line call to join instead of a for loop Updated black, mirrors-prettier, and detect-secrets in pre-commit hooks Made root_dispatch_id nullable to circumvent migration issues with sqlite in certain platforms ### Removed Postprocessing from runners. References to specific IBMQ hub/group/project in tutorial 5 Slurm executor reference from qaoa tutorial since it's not compatible with conda env at the moment. ### Fixed Doubling of nodes that are added to the transport graph. Ensure postprocessing node end time is added as the workflow end time. Functional tests Custom executor how to guide, and its loading mechanism in covalent server. Broken postprocessing unit test. UI crashing if time values are null No longer adding "http://" every time a dispatcher address is provided in local dispatcher class in order to use the provided address exactly Redispatch bug involving copying reusable nodes from old transport graph to new transport graph. Pennylane tutorial notebook. Redispatch bug. Braket pip installation instructions. Removing the entrypoint for SDK-only install Updating client requirements to match server versions Load plugins only when COVALENT_PLUGIN_LOAD environment variable has been set to a Truthy value. MNIST tutorial now shows non-Null outputs and the classifier training log image has been updated. Minor changes to tutorials: autoencoder, quantum and classical svm, ensemble classification, iris classification with Pennylane, quantum chemistry, DNN tutorial, qaoa, spacetime tutorial etc. The range of networkx versions in requirements.txt weren't compatible with each other, thus it is pinned to 2.8.6 now SDK-only sdist and installation should now work as expected, not packaging the server Fixed naming of collection nodes (was breaking postprocessing) Restored compatibility with stable release of AWS executors Optimization of logs on the GUI for large log file sizes. Fixed UI pagination not working for more than 11 pages Runtime field counting down for select running dispatches Removed merge conflict symbols in changelog Build graph now sets all unset lattice constraints from defaults Fixed all failing functional tests Fixed local executor tests on MacOS by adding ProcessPoolExecutor Respecting specified AWS profile & region in remote executed S3 file transfers, defaulting to env vars of execution backend Electrons will inherit the lattice executors. Sublattices inherit the parent lattice executor. When several electrons are running concurrently, their stdout and stderr are stored in the correct graph nodes. Electron errors now appear in the Electron ErrorCard when one clicks on a failed task in the UI. When an electron raises an exception during execution, the local and dask executors now try to recover any output that was already written. Fixed functional tests. Added requirements-client.txt to MANIFEST file Respecting specified AWS profile & region in remote executed S3 file transfers, defaulting to env vars of execution backend Fixed local executor tests on MacOS (second attempt) The initialize_results_dir method attempts to use an environment variable instead of the results directory in the payload Modified certain sqlalchemy commands for postgres compatibility Removed references to results_dir in the payload Folder transfer unit test. Folder transfer download bug Result objects now print correctly when nodes fail Enabling logging by default Removed debugging output Clarify cli output when covalent db migrate needs to be run Rendering newlines in ErrorCard on the UI for displaying error stacktraces VERSION incrementing logic in changelog Fixed v11 migration to use render as batch to make DROP operations compatible with sqlite covalent restart honors the sdk.no_cluster setting Re-applying default executor fix post config file reunification Exceptions when instantiating executors are handled Covalent start now waits for the server to settle before returning ### Tests Updated and added tests to account for all of the above triggers related changes

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.