I started off the week by upgrading mypy package to 0.511 and running mypy on schema_salad codebase without any errors. Alekxandr has been really helpful in getting this done. He wrote mistune package stub file and also raised an issue at ruamel repo, regarding the errors generated by ruamel.yaml package when running under mypy==0.511. Anthon made a quick patch for the same. This reduced the total number of errors but unfortuantely there were still a few coming up when running mypy for schema_salad, so I ended up writing mypy.ini file to specifically silence ruamel package errors, for the time being.

After this I made some progress in running mypy without --py2 flag i.e. mypy will treat the codebase as python3 compatible and search for errors with this assumption. First step was to make sure I have all the stub files for the dependency packages before running mypy. A lot of stdlib and 3rd party packages are now availble on python/typeshed repo and come bundled with mypy. For the ones, that were not there, I ended up using stubgen utility to autogenerate stub files and refactored stub files written for py2 mypy to work for py3 checking as well. In future, I might write better annotated stub files for specific packages, if needed.
Ideally, it’s better to comment out imports from .pyi stubs that are not needed. The goal is represent the public API, or at least the part we use. My PR for mypy updates for schema_salad: schema_salad/pull/111

In cwltool, I hit a roadblock in porting after encountering various errors related to Exceptions raised by avro-python3 package. I tried setting up breakpoints and comparing value of the variables in the stack just before the Exception occurs, for py2 and py3 runtimes, but that didn’t help. I spent a few hours on this one, before reporting the errors on GitHub so someone else can help me with it.

In the end, I did some minor refactoring to move few helper functions to utils.py file.

ToDo Week 3:

  • Run mypy without --py2 flag in schema_salad with no errros
  • Resolve roadblock errors in cwltool porting
  • Might start working on cwltool mypy package upgrade.