protocol/python-packages/parallel_without_sra_client
F. Eugene Aumson ec807120c3
Migrate to Web3.py v5 (#2038)
* Install Py packages in dep. order, not in parallel

Install Python packages in dependency order, not in parallel.

* sra_client.py: Add `./setup.py clean`

* Fix python package dependency ordering...

...and include a script to produce the proper ordering.

* sra_client.py: reformat whitespace in doctest

* contract_wrappers.py: don't auto-import wrappers

This was discovered while minimizing CircleCI steps to dianose a problem
with running the Launch Kit Backend in CircleCI.

These classes should be imported via the
zero_ex.contract_wrappers.exchange and
zero_ex.contract_wrappers.erc20_token modules, respectively.  We
permitted importing them from just zero_ex.contract_wrappers back when
they were the only wrappers we had, but now that we have so many
different contracts being wrapped, this is just another list to keep
manually updated (which, obviously is error prone, since it slipped
through the cracks already), so it's better to just not support this
type of import.

* abi-gen/Py: doc contract method attributes

Without this, generated documentation was not including the class
members that represent the contract methods, rendering the usage
unclear.

* sra_client.py: disable tests in CI

* abi-gen/Py: strip repeated spaces from devdoc

* contract_wrappers.py: gen docs for all wrappers...

...except for the dummy tokens.

* sra_client.py/test: change launch kit docker image

Previously these teses were using 0xorg/launch-kit-ci, but that was a
one-off thing created just for CI, back before there was a regularly
maintained docker image of Launch Kit.

Changed to use 0xorg/launch-kit-backend since it's regularly
maintained/updated.

Because the -backend image is using a different Linux distribution
(Alpine), the commands used to wait for ganache startup also had to
change.

The tag used, 74bcc39, is provisional due to the pending Issue at
https://github.com/0xProject/0x-launch-kit-backend/issues/73 .  When
that issue is resolved, the tag suffix on the imag name should be
removed.

* Migrate from Web3.py 4.x to 5.x

* sra_client.py: checksum address in doctest

Due to problem with launch-kit-backend, documented at
https://github.com/0xProject/0x-launch-kit-backend/issues/73 ,
we need to checksum the makerAddress, in the order retrieved from the
relayer, before filling it, otherwise Web3.py gives this error:

InvalidAddress('Web3.py only accepts checksum addresses. The software
that gave you this non-checksum address should be considered unsafe,
please file it as a bug on their platform. Try using an ENS name
instead. Or, if you must accept lower safety, use
Web3.toChecksumAddress(lower_case_address).',
'0x5409ed021d9299bf6814279a6a1411a7e866a631')

* Update CHANGELOGs

* sra_client.py: make CHANGELOG be REVESE chrono.

Formerly CHANGELOG was in chronological order.  Now it's in reverse
chronological order.

* abi-gen/Py: fix missing space in sanitized devdoc
2019-08-08 14:53:59 -04:00

55 lines
1.7 KiB
Python
Executable File

#!/usr/bin/env python
"""Run the given command in all packages in parallel.
Handy for quick verification test runs, but annoying in that all of the output
is interleaved.
$ ./parallel ./setup.py lint
This will `cd` into each package, run `./setup.py lint`, then `cd ..`, all in
parallel, in a separate process for each package. The number of processes is
decided by ProcessPoolExecutor. Replace "lint" with any of "test", "clean",
"build_sphinx" (for docs), etc.
Also consider:
$ ./parallel pip install -e .[dev] # install all the packages in editable mode
$ ./parallel pip uninstall $(basename $(pwd))
>>>"""
from concurrent.futures import ProcessPoolExecutor, wait
from os import chdir
from subprocess import CalledProcessError, check_output
from sys import argv
PACKAGES = [
"contract_addresses",
"contract_artifacts",
"contract_wrappers",
"json_schemas",
"order_utils",
"middlewares",
]
def run_cmd_on_package(package: str):
"""cd to the package dir, ./setup.py lint, cd .."""
chdir(package)
try:
check_output(f"{' '.join(argv[1:])}".split())
except CalledProcessError as error:
print(f"standard output from command:\n{error.output.decode('utf-8')}")
raise RuntimeError(f"Above exception raised in {package}, ") from error
finally:
chdir("..")
with ProcessPoolExecutor() as executor:
for future in executor.map(run_cmd_on_package, PACKAGES):
# iterate over map()'s return value, to resolve the futures.
# but we don't actually care what the return values are, so just `pass`.
# if any exceptions were raised by the underlying task, they'll be
# raised as the iteration encounters them.
pass