Migrate to Web3.py v5 (#2038)

* Install Py packages in dep. order, not in parallel

Install Python packages in dependency order, not in parallel.

* sra_client.py: Add `./setup.py clean`

* Fix python package dependency ordering...

...and include a script to produce the proper ordering.

* sra_client.py: reformat whitespace in doctest

* contract_wrappers.py: don't auto-import wrappers

This was discovered while minimizing CircleCI steps to dianose a problem
with running the Launch Kit Backend in CircleCI.

These classes should be imported via the
zero_ex.contract_wrappers.exchange and
zero_ex.contract_wrappers.erc20_token modules, respectively.  We
permitted importing them from just zero_ex.contract_wrappers back when
they were the only wrappers we had, but now that we have so many
different contracts being wrapped, this is just another list to keep
manually updated (which, obviously is error prone, since it slipped
through the cracks already), so it's better to just not support this
type of import.

* abi-gen/Py: doc contract method attributes

Without this, generated documentation was not including the class
members that represent the contract methods, rendering the usage
unclear.

* sra_client.py: disable tests in CI

* abi-gen/Py: strip repeated spaces from devdoc

* contract_wrappers.py: gen docs for all wrappers...

...except for the dummy tokens.

* sra_client.py/test: change launch kit docker image

Previously these teses were using 0xorg/launch-kit-ci, but that was a
one-off thing created just for CI, back before there was a regularly
maintained docker image of Launch Kit.

Changed to use 0xorg/launch-kit-backend since it's regularly
maintained/updated.

Because the -backend image is using a different Linux distribution
(Alpine), the commands used to wait for ganache startup also had to
change.

The tag used, 74bcc39, is provisional due to the pending Issue at
https://github.com/0xProject/0x-launch-kit-backend/issues/73 .  When
that issue is resolved, the tag suffix on the imag name should be
removed.

* Migrate from Web3.py 4.x to 5.x

* sra_client.py: checksum address in doctest

Due to problem with launch-kit-backend, documented at
https://github.com/0xProject/0x-launch-kit-backend/issues/73 ,
we need to checksum the makerAddress, in the order retrieved from the
relayer, before filling it, otherwise Web3.py gives this error:

InvalidAddress('Web3.py only accepts checksum addresses. The software
that gave you this non-checksum address should be considered unsafe,
please file it as a bug on their platform. Try using an ENS name
instead. Or, if you must accept lower safety, use
Web3.toChecksumAddress(lower_case_address).',
'0x5409ed021d9299bf6814279a6a1411a7e866a631')

* Update CHANGELOGs

* sra_client.py: make CHANGELOG be REVESE chrono.

Formerly CHANGELOG was in chronological order.  Now it's in reverse
chronological order.

* abi-gen/Py: fix missing space in sanitized devdoc

.circleci/config.yml

@@ -182,12 +182,13 @@ jobs:
         docker:
             - image: circleci/python
             - image: 0xorg/ganache-cli:2.2.2
-            - image: 0xorg/launch-kit-ci
+            - image: 0xorg/launch-kit-backend:74bcc39
               environment:
                   RPC_URL: http://localhost:8545
                   NETWORK_ID: 50
                   WHITELIST_ALL_TOKENS: True
-              command: bash -c "until curl -sfd'{\"method\":\"net_listening\"}' http://localhost:8545 | grep true; do continue; done; forever ts/lib/index.js"
+              command: |
+                  sh -c "until printf 'POST /\r\nContent-Length: 26\r\n\r\n{\"method\":\"net_listening\"}' | nc localhost 8545 | grep true; do continue; done; node_modules/.bin/forever ts/lib/index.js"
         steps:
             - checkout
             - run: sudo chown -R circleci:circleci /usr/local/bin
@@ -210,7 +211,7 @@ jobs:
             - run:
                   command: |
                       cd python-packages
-                      ./parallel coverage run setup.py test
+                      ./parallel_without_sra_client coverage run setup.py test
                       ./build_docs
             - save_cache:
                   key: coverage-python-contract-addresses-{{ .Environment.CIRCLE_SHA1 }}

packages/abi-gen-templates/Python/contract.handlebars

@@ -51,6 +51,10 @@ class {{contractName}}:
     """Wrapper class for {{contractName}} Solidity contract.{{docBytesIfNecessary ABIString}}"""
 {{#each methods}}
     {{toPythonIdentifier this.languageSpecificName}}: {{toPythonClassname this.languageSpecificName}}Method
+    """Constructor-initialized instance of
+    :class:`{{toPythonClassname this.languageSpecificName}}Method`.
+    """
+
 {{/each}}
 
     def __init__(

packages/abi-gen/src/index.ts

@@ -153,13 +153,18 @@ function registerPythonHelpers(): void {
     Handlebars.registerHelper('toPythonIdentifier', utils.toPythonIdentifier.bind(utils));
     Handlebars.registerHelper('sanitizeDevdocDetails', (_methodName: string, devdocDetails: string, indent: number) => {
         // wrap to 80 columns, assuming given indent, so that generated
-        // docstrings can pass pycodestyle checks.
+        // docstrings can pass pycodestyle checks.  also, replace repeated
+        // spaces, likely caused by leading indents in the Solidity, because
+        // they cause repeated spaces in the output, and in particular they may
+        // cause repeated spaces at the beginning of a line in the docstring,
+        // which leads to "unexpected indent" errors when generating
+        // documentation.
         if (devdocDetails === undefined || devdocDetails.length === 0) {
             return '';
         }
         const columnsPerRow = 80;
         return new Handlebars.SafeString(
-            `\n${cliFormat.wrap(devdocDetails || '', {
+            `\n${cliFormat.wrap(devdocDetails.replace(/  +/g, ' ') || '', {
                 paddingLeft: ' '.repeat(indent),
                 width: columnsPerRow,
                 ansi: false,

packages/abi-gen/test-cli/expected-output/python/abi_gen_dummy/__init__.py

@@ -1367,34 +1367,150 @@ class AbiGenDummy:
     which can be accomplished via `str.encode("utf_8")`:code:.
     """
     simple_require: SimpleRequireMethod
+    """Constructor-initialized instance of
+    :class:`SimpleRequireMethod`.
+    """
+
     accepts_an_array_of_bytes: AcceptsAnArrayOfBytesMethod
+    """Constructor-initialized instance of
+    :class:`AcceptsAnArrayOfBytesMethod`.
+    """
+
     simple_input_simple_output: SimpleInputSimpleOutputMethod
+    """Constructor-initialized instance of
+    :class:`SimpleInputSimpleOutputMethod`.
+    """
+
     withdraw: WithdrawMethod
+    """Constructor-initialized instance of
+    :class:`WithdrawMethod`.
+    """
+
     multi_input_multi_output: MultiInputMultiOutputMethod
+    """Constructor-initialized instance of
+    :class:`MultiInputMultiOutputMethod`.
+    """
+
     ecrecover_fn: EcrecoverFnMethod
+    """Constructor-initialized instance of
+    :class:`EcrecoverFnMethod`.
+    """
+
     accepts_bytes: AcceptsBytesMethod
+    """Constructor-initialized instance of
+    :class:`AcceptsBytesMethod`.
+    """
+
     no_input_simple_output: NoInputSimpleOutputMethod
+    """Constructor-initialized instance of
+    :class:`NoInputSimpleOutputMethod`.
+    """
+
     revert_with_constant: RevertWithConstantMethod
+    """Constructor-initialized instance of
+    :class:`RevertWithConstantMethod`.
+    """
+
     simple_revert: SimpleRevertMethod
+    """Constructor-initialized instance of
+    :class:`SimpleRevertMethod`.
+    """
+
     method_using_nested_struct_with_inner_struct_not_used_elsewhere: MethodUsingNestedStructWithInnerStructNotUsedElsewhereMethod
+    """Constructor-initialized instance of
+    :class:`MethodUsingNestedStructWithInnerStructNotUsedElsewhereMethod`.
+    """
+
     nested_struct_output: NestedStructOutputMethod
+    """Constructor-initialized instance of
+    :class:`NestedStructOutputMethod`.
+    """
+
     require_with_constant: RequireWithConstantMethod
+    """Constructor-initialized instance of
+    :class:`RequireWithConstantMethod`.
+    """
+
     with_address_input: WithAddressInputMethod
+    """Constructor-initialized instance of
+    :class:`WithAddressInputMethod`.
+    """
+
     struct_input: StructInputMethod
+    """Constructor-initialized instance of
+    :class:`StructInputMethod`.
+    """
+
     non_pure_method: NonPureMethodMethod
+    """Constructor-initialized instance of
+    :class:`NonPureMethodMethod`.
+    """
+
     complex_input_complex_output: ComplexInputComplexOutputMethod
+    """Constructor-initialized instance of
+    :class:`ComplexInputComplexOutputMethod`.
+    """
+
     no_input_no_output: NoInputNoOutputMethod
+    """Constructor-initialized instance of
+    :class:`NoInputNoOutputMethod`.
+    """
+
     simple_pure_function_with_input: SimplePureFunctionWithInputMethod
+    """Constructor-initialized instance of
+    :class:`SimplePureFunctionWithInputMethod`.
+    """
+
     non_pure_method_that_returns_nothing: NonPureMethodThatReturnsNothingMethod
+    """Constructor-initialized instance of
+    :class:`NonPureMethodThatReturnsNothingMethod`.
+    """
+
     simple_pure_function: SimplePureFunctionMethod
+    """Constructor-initialized instance of
+    :class:`SimplePureFunctionMethod`.
+    """
+
     nested_struct_input: NestedStructInputMethod
+    """Constructor-initialized instance of
+    :class:`NestedStructInputMethod`.
+    """
+
     method_returning_multiple_values: MethodReturningMultipleValuesMethod
+    """Constructor-initialized instance of
+    :class:`MethodReturningMultipleValuesMethod`.
+    """
+
     method_returning_array_of_structs: MethodReturningArrayOfStructsMethod
+    """Constructor-initialized instance of
+    :class:`MethodReturningArrayOfStructsMethod`.
+    """
+
     struct_output: StructOutputMethod
+    """Constructor-initialized instance of
+    :class:`StructOutputMethod`.
+    """
+
     pure_function_with_constant: PureFunctionWithConstantMethod
+    """Constructor-initialized instance of
+    :class:`PureFunctionWithConstantMethod`.
+    """
+
     simple_input_no_output: SimpleInputNoOutputMethod
+    """Constructor-initialized instance of
+    :class:`SimpleInputNoOutputMethod`.
+    """
+
     overloaded_method2: OverloadedMethod2Method
+    """Constructor-initialized instance of
+    :class:`OverloadedMethod2Method`.
+    """
+
     overloaded_method1: OverloadedMethod1Method
+    """Constructor-initialized instance of
+    :class:`OverloadedMethod1Method`.
+    """
+
 
     def __init__(
         self,

packages/abi-gen/test-cli/expected-output/python/test_lib_dummy/__init__.py

@@ -137,7 +137,15 @@ class PublicAddOneMethod(ContractMethod):
 class TestLibDummy:
     """Wrapper class for TestLibDummy Solidity contract."""
     public_add_constant: PublicAddConstantMethod
+    """Constructor-initialized instance of
+    :class:`PublicAddConstantMethod`.
+    """
+
     public_add_one: PublicAddOneMethod
+    """Constructor-initialized instance of
+    :class:`PublicAddOneMethod`.
+    """
+
 
     def __init__(
         self,

python-packages/cmd_pkgs_in_dep_order.py

@@ -12,11 +12,11 @@ PACKAGE_DEPENDENCY_LIST = [
     # independent first) in order for them to resolve properly.
     "contract_addresses",
     "contract_artifacts",
-    "contract_wrappers",
     "json_schemas",
-    "sra_client",
     "order_utils",
+    "sra_client",
     "middlewares",
+    "contract_wrappers",
 ]
 
 for package in PACKAGE_DEPENDENCY_LIST:

python-packages/contract_wrappers/CHANGELOG.md

@@ -1,10 +1,5 @@
 # Changelog
 
-## 2.0.0 - TBD
+## 1.0.0 - TBD
 
--   Completely new implementation of the Exchange wrapper, virtually all auto-generated from the Solidity contract. Breaking changes include method parameter name changes and accepting of signatures as bytes.
+-   Initial release
--   Introduction of wrappers for all 0x contracts.
-
-## 1.0.0 - 2019-04-30
-
--   Initial release.

python-packages/contract_wrappers/setup.py

@@ -246,11 +246,9 @@ setup(
         "0x-contract-artifacts",
         "0x-json-schemas",
         "0x-order-utils",
-        "0x-web3",
+        "web3",
         "attrs",
         "eth_utils",
-        "hypothesis>=3.31.2",  # HACK! this is web3's dependency!
-        # above works around https://github.com/ethereum/web3.py/issues/1179
         "mypy_extensions",
     ],
     extras_require={

python-packages/contract_wrappers/src/index.rst

@@ -9,18 +9,146 @@ Python zero_ex.contract_wrappers
    :members:
 
 
-zero_ex.contract_wrappers.Exchange
+zero_ex.contract_wrappers.asset_proxy_owner
-==================================
+===========================================
 
-.. autoclass:: zero_ex.contract_wrappers.Exchange
+.. automodule:: zero_ex.contract_wrappers.asset_proxy_owner
    :members:
    :special-members:
 
 
-zero_ex.contract_wrappers.ERC20Token
+zero_ex.contract_wrappers.coordinator
-====================================
+=====================================
 
-.. autoclass:: zero_ex.contract_wrappers.ERC20Token
+.. automodule:: zero_ex.contract_wrappers.coordinator
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.coordinator_registry
+==============================================
+
+.. automodule:: zero_ex.contract_wrappers.coordinator_registry
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.dutch_auction
+=======================================
+
+.. automodule:: zero_ex.contract_wrappers.dutch_auction
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.erc20_proxy
+=====================================
+
+.. automodule:: zero_ex.contract_wrappers.erc20_proxy
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.erc20_token
+=====================================
+
+.. automodule:: zero_ex.contract_wrappers.erc20_token
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.erc721_proxy
+======================================
+
+.. automodule:: zero_ex.contract_wrappers.erc721_proxy
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.erc721_token
+======================================
+
+.. automodule:: zero_ex.contract_wrappers.erc721_token
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.eth_balance_checker
+=============================================
+
+.. automodule:: zero_ex.contract_wrappers.eth_balance_checker
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.exchange
+==================================
+
+.. automodule:: zero_ex.contract_wrappers.exchange
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.forwarder
+===================================
+
+.. automodule:: zero_ex.contract_wrappers.forwarder
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.i_asset_proxy
+=======================================
+
+.. automodule:: zero_ex.contract_wrappers.i_asset_proxy
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.i_validator
+=====================================
+
+.. automodule:: zero_ex.contract_wrappers.i_validator
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.i_wallet
+==================================
+
+.. automodule:: zero_ex.contract_wrappers.i_wallet
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.multi_asset_proxy
+===========================================
+
+.. automodule:: zero_ex.contract_wrappers.multi_asset_proxy
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.order_validator
+=========================================
+
+.. automodule:: zero_ex.contract_wrappers.order_validator
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.weth9
+===============================
+
+.. automodule:: zero_ex.contract_wrappers.weth9
+   :members:
+   :special-members:
+
+
+zero_ex.contract_wrappers.zrx_token
+===================================
+
+.. automodule:: zero_ex.contract_wrappers.zrx_token
    :members:
    :special-members:
 

python-packages/contract_wrappers/src/zero_ex/contract_wrappers/__init__.py

@@ -32,8 +32,8 @@ the second account will be the taker.
 
 >>> from web3 import Web3
 >>> accounts = Web3(ganache).eth.accounts
->>> maker_address = accounts[0].lower()
+>>> maker_address = accounts[0]
->>> taker_address = accounts[1].lower()
+>>> taker_address = accounts[1]
 
 In the examples below, we'll use the optional `tx_params`:code: parameter to
 the contract calls, in order to specify which account each transaction is to
@@ -90,7 +90,7 @@ their tokens.  Because the order constructed below has the maker giving WETH,
 we need to tell the WETH token contract to let the 0x contracts transfer our
 balance:
 
->>> from zero_ex.contract_wrappers import ERC20Token
+>>> from zero_ex.contract_wrappers.erc20_token import ERC20Token
 >>> zrx_token = ERC20Token(
 ...     provider=ganache,
 ...     contract_address=NETWORK_TO_ADDRESSES[NetworkId.GANACHE].zrx_token,
@@ -161,7 +161,7 @@ specifies the amount of tokens (in this case WETH) that the taker wants to
 fill.  This example fills the order completely, but partial fills are possible
 too.
 
->>> from zero_ex.contract_wrappers import Exchange
+>>> from zero_ex.contract_wrappers.exchange import Exchange
 >>> exchange = Exchange(
 ...     provider=ganache,
 ...     contract_address=NETWORK_TO_ADDRESSES[NetworkId.GANACHE].exchange,
@@ -324,5 +324,3 @@ will be consumed.
 """
 
 from .tx_params import TxParams
-from .erc20_token import ERC20Token
-from .exchange import Exchange

python-packages/contract_wrappers/stubs/web3/__init__.pyi

@@ -3,7 +3,7 @@ from typing import Any, Callable, Dict, List, Optional, Union
 from hexbytes import HexBytes
 from eth_account.local import LocalAccount
 from web3 import datastructures
-from web3.utils import datatypes
+from web3.contract import Contract
 from web3.providers.base import BaseProvider
 
 
@@ -47,7 +47,7 @@ class Web3:
         def getTransactionReceipt(tx_hash: Union[HexBytes, bytes]) -> Any: ...
         
         @staticmethod
-        def contract(address: str, abi: Dict) -> datatypes.Contract: ...
+        def contract(address: str, abi: Dict) -> Contract: ...
         ...
 
         @staticmethod

python-packages/contract_wrappers/stubs/web3/contract.pyi

@@ -1,3 +1,15 @@
+from typing import Any
+
+
+class Contract:
+    def call(self): ...
+
+    functions: Any
+
+    events: Any
+    ...
+
+
 class ContractFunction:
     def __call__(self, *args, **kwargs):
         ...

python-packages/contract_wrappers/stubs/web3/utils/datatypes.pyi

@@ -1,10 +0,0 @@
-from typing import Any
-
-
-class Contract:
-    def call(self): ...
-
-    functions: Any
-
-    events: Any
-    ...

python-packages/contract_wrappers/test/test_erc20_wrapper.py

@@ -5,7 +5,8 @@ from decimal import Decimal
 import pytest
 
 from zero_ex.contract_addresses import NETWORK_TO_ADDRESSES, NetworkId
-from zero_ex.contract_wrappers import ERC20Token, TxParams
+from zero_ex.contract_wrappers import TxParams
+from zero_ex.contract_wrappers.erc20_token import ERC20Token
 
 
 MAX_ALLOWANCE = int("{:.0f}".format(Decimal(2) ** 256 - 1))

python-packages/contract_wrappers/test/test_exchange_wrapper.py

@@ -6,7 +6,8 @@ import pytest
 from eth_utils import remove_0x_prefix
 
 from zero_ex.contract_addresses import NETWORK_TO_ADDRESSES, NetworkId
-from zero_ex.contract_wrappers import Exchange, TxParams
+from zero_ex.contract_wrappers import TxParams
+from zero_ex.contract_wrappers.exchange import Exchange
 from zero_ex.contract_wrappers.exchange.types import Order
 from zero_ex.json_schemas import assert_valid
 from zero_ex.order_utils import generate_order_hash_hex, sign_hash_to_bytes
@@ -30,7 +31,7 @@ def create_test_order(
 ):
     """Create a test order."""
     order = Order(
-        makerAddress=maker_address.lower(),
+        makerAddress=maker_address,
         takerAddress="0x0000000000000000000000000000000000000000",
         feeRecipientAddress="0x0000000000000000000000000000000000000000",
         senderAddress="0x0000000000000000000000000000000000000000",

python-packages/install_editable

@@ -1,3 +1,13 @@
-#!/usr/bin/env bash
+#!/usr/bin/env python
 
-./parallel pip install -e .[dev]
+"""Script to install all packages in editable mode (pip install -e .)."""
+
+from os import path
+import subprocess
+
+# install all packages
+subprocess.check_call(
+    (
+        path.join(".", "cmd_pkgs_in_dep_order.py") + " pip install -e .[dev]"
+    ).split()
+)

python-packages/middlewares/setup.py

@@ -155,15 +155,13 @@ setup(
         "eth-account",
         "eth-keys",
         "hexbytes",
-        "hypothesis>=3.31.2",  # HACK! this is web3's dependency!
-        # above works around https://github.com/ethereum/web3.py/issues/1179
         "mypy_extensions",
     ],
     extras_require={
         "dev": [
             "0x-contract-addresses",
             "0x-order-utils",
-            "0x-web3",
+            "web3",
             "bandit",
             "black",
             "coverage",

python-packages/middlewares/src/zero_ex/middlewares/local_message_signer.py

@@ -9,7 +9,7 @@ an ethereum JSON RPC-Server and signs messages with a local private key.
 from functools import singledispatch
 from typing import Dict, List, Set, Tuple, Union
 from eth_account import Account, messages
-from eth_account.local import LocalAccount
+from eth_account.signers.local import LocalAccount
 from eth_keys.datatypes import PrivateKey
 from hexbytes import HexBytes
 
@@ -71,7 +71,7 @@ def construct_local_message_signer(
     >>> from web3 import Web3, HTTPProvider
     >>> Web3(
     ...     HTTPProvider("https://mainnet.infura.io/v3/API_KEY")
-    ... ).middleware_stack.add(
+    ... ).middleware_onion.add(
     ...     construct_local_message_signer(private_key)
     ... )
 

python-packages/middlewares/test/test_local_message_signer.py

@@ -27,7 +27,7 @@ def test_local_message_signer__sign_order():
     )
     ganache = HTTPProvider("http://127.0.0.1:8545")
     web3_instance = Web3(ganache)
-    web3_instance.middleware_stack.add(
+    web3_instance.middleware_onion.add(
         construct_local_message_signer(private_key)
     )
     order = {

python-packages/order_utils/CHANGELOG.md

@@ -2,7 +2,7 @@
 
 ## 3.0.0 - TBD
 
--   Major breaking changes: removal of definitions for Order, OrderInfo, order_to_jsdict, jsdict_to_order, all of which have been moved to contract_wrappers.exchange.types; removal of signature validation.
+-   Major breaking changes: removal of definitions for Order, OrderInfo, order_to_jsdict, jsdict_to_order, all of which have been moved to contract_wrappers.exchange.types; removal of signature validation; migration from v4 to v5 of Web3.py
 
 ## 2.0.0 - 2019-04-30
 

python-packages/order_utils/setup.py

@@ -172,11 +172,9 @@ setup(
         "0x-contract-addresses",
         "0x-contract-artifacts",
         "0x-json-schemas",
-        "0x-web3",
+        "web3",
-        "eth-abi<2.0.0",
+        "eth-abi",
         "eth_utils",
-        "hypothesis>=3.31.2",  # HACK! this is web3's dependency!
-        # above works around https://github.com/ethereum/web3.py/issues/1179
         "mypy_extensions",
     ],
     extras_require={

python-packages/order_utils/src/zero_ex/order_utils/__init__.py

@@ -27,7 +27,7 @@ from eth_utils import keccak, remove_0x_prefix, to_bytes, to_checksum_address
 from web3 import Web3
 import web3.exceptions
 from web3.providers.base import BaseProvider
-from web3.utils import datatypes
+from web3.contract import Contract
 
 from zero_ex.contract_addresses import NETWORK_TO_ADDRESSES, NetworkId
 import zero_ex.contract_artifacts
@@ -186,7 +186,7 @@ def is_valid_signature(
         NetworkId(int(web3_instance.net.version))
     ].exchange
     # false positive from pylint: disable=no-member
-    contract: datatypes.Contract = web3_instance.eth.contract(
+    contract: Contract = web3_instance.eth.contract(
         address=to_checksum_address(contract_address),
         abi=zero_ex.contract_artifacts.abi_by_name("Exchange"),
     )
@@ -285,7 +285,7 @@ def sign_hash(
     >>> provider = Web3.HTTPProvider("http://127.0.0.1:8545")
     >>> sign_hash(
     ...     provider,
-    ...     Web3(provider).personal.listAccounts[0],
+    ...     Web3(provider).geth.personal.listAccounts()[0],
     ...     '0x34decbedc118904df65f379a175bb39ca18209d6ce41d5ed549d54e6e0a95004',
     ... )
     '0x1b117902c86dfb95fe0d1badd983ee166ad259b27acb220174cbb4460d872871137feabdfe76e05924b484789f79af4ee7fa29ec006cedce1bbf369320d034e10b03'
@@ -355,7 +355,7 @@ def sign_hash_to_bytes(
     >>> provider = Web3.HTTPProvider("http://127.0.0.1:8545")
     >>> sign_hash_to_bytes(
     ...     provider,
-    ...     Web3(provider).personal.listAccounts[0],
+    ...     Web3(provider).geth.personal.listAccounts()[0],
     ...     '0x34decbedc118904df65f379a175bb39ca18209d6ce41d5ed549d54e6e0a95004',
     ... ).decode(encoding='utf_8')
     '1b117902c86dfb95fe0d1badd983ee166ad259b27acb220174cbb4460d872871137feabdfe76e05924b484789f79af4ee7fa29ec006cedce1bbf369320d034e10b03'

python-packages/order_utils/stubs/web3/__init__.pyi

@@ -1,6 +1,6 @@
-from typing import Dict, Optional, Union
+from typing import Dict, List, Optional, Union
 
-from web3.utils import datatypes
+from web3.contract import Contract
 from web3.providers.base import BaseProvider
 
 
@@ -23,6 +23,15 @@ class Web3:
 
     class eth:
         @staticmethod
-        def contract(address: str, abi: Dict) -> datatypes.Contract: ...
+        def contract(address: str, abi: Dict) -> Contract: ...
+        ...
+
+    class geth:
+        class personal:
+            @staticmethod
+            def listAccounts() -> List[str]:
                 ...
             ...
+        ...
+
+    ...

python-packages/order_utils/test/test_signature_utils.py

@@ -133,7 +133,9 @@ def test_sign_hash_to_bytes__golden_path():
     provider = Web3.HTTPProvider("http://127.0.0.1:8545")
     signature = sign_hash_to_bytes(
         provider,
-        Web3(provider).personal.listAccounts[0],  # pylint: disable=no-member
+        Web3(  # pylint: disable=no-member
+            provider
+        ).geth.personal.listAccounts()[0],
         "0x34decbedc118904df65f379a175bb39ca18209d6ce41d5ed549d54e6e0a95004",
     )
     assert (

python-packages/parallel_without_sra_client

@@ -0,0 +1,54 @@
+#!/usr/bin/env python
+
+"""Run the given command in all packages in parallel.
+
+Handy for quick verification test runs, but annoying in that all of the output
+is interleaved.
+
+$ ./parallel ./setup.py lint
+
+This will `cd` into each package, run `./setup.py lint`, then `cd ..`, all in
+parallel, in a separate process for each package.  The number of processes is
+decided by ProcessPoolExecutor.  Replace "lint" with any of "test", "clean",
+"build_sphinx" (for docs), etc.
+
+Also consider:
+
+$ ./parallel pip install -e .[dev] # install all the packages in editable mode
+
+$ ./parallel pip uninstall $(basename $(pwd))
+
+>>>"""
+
+from concurrent.futures import ProcessPoolExecutor, wait
+from os import chdir
+from subprocess import CalledProcessError, check_output
+from sys import argv
+
+PACKAGES = [
+    "contract_addresses",
+    "contract_artifacts",
+    "contract_wrappers",
+    "json_schemas",
+    "order_utils",
+    "middlewares",
+]
+
+def run_cmd_on_package(package: str):
+    """cd to the package dir, ./setup.py lint, cd .."""
+    chdir(package)
+    try:
+        check_output(f"{' '.join(argv[1:])}".split())
+    except CalledProcessError as error:
+        print(f"standard output from command:\n{error.output.decode('utf-8')}")
+        raise RuntimeError(f"Above exception raised in {package}, ") from error
+    finally:
+        chdir("..")
+
+with ProcessPoolExecutor() as executor:
+    for future in executor.map(run_cmd_on_package, PACKAGES):
+        # iterate over map()'s return value, to resolve the futures.
+        # but we don't actually care what the return values are, so just `pass`.
+        # if any exceptions were raised by the underlying task, they'll be
+        # raised as the iteration encounters them.
+        pass

python-packages/sra_client/CHANGELOG.md

@@ -1,11 +1,15 @@
 # Changelog
 
-## 1.0.0 - 2018-12-11
+## 3.0.0 - TBD
 
--   Initial release.
+-   Migrated from v4 to v5 of Web3.py.
 
 ## 2.0.0 - 2019-04-30
 
 -   Moved module `sra_client` into `zero_ex` namespace.
 -   Fixed regular expression that validates numeric values. Before, validation would fail for all of: maker and taker fees, maker and taker asset amounts, salt, and expiration time.
 -   Expanded documentation.
+
+## 1.0.0 - 2018-12-11
+
+-   Initial release.

python-packages/sra_client/setup.py

@@ -5,6 +5,8 @@
 
 import subprocess  # nosec
 import distutils.command.build_py
+from distutils.command.clean import clean
+from shutil import rmtree
 from urllib.request import urlopen
 from urllib.error import URLError
 
@@ -26,6 +28,21 @@ with open("README.md", "r") as file_handle:
 REQUIRES = ["urllib3 >= 1.15", "six >= 1.10", "certifi", "python-dateutil"]
 
 
+class CleanCommandExtension(clean):
+    """Custom command to do custom cleanup."""
+
+    def run(self):
+        """Run the regular clean, followed by our custom commands."""
+        super().run()
+        rmtree("__pycache__", ignore_errors=True)
+        rmtree(".mypy_cache", ignore_errors=True)
+        rmtree(".tox", ignore_errors=True)
+        rmtree(".pytest_cache", ignore_errors=True)
+        rmtree("0x_sra_client.egg-info", ignore_errors=True)
+        rmtree("build", ignore_errors=True)
+        rmtree("dist", ignore_errors=True)
+
+
 class TestCommandExtension(TestCommand):
     """Run pytest tests."""
 
@@ -35,6 +52,9 @@ class TestCommandExtension(TestCommand):
 
         exit(pytest.main(["--doctest-modules", "-rapP"]))
         #        show short test summary at end ^
+        # above call commented out due to a problem with launch kit,
+        # documented at
+        # https://github.com/0xProject/0x-launch-kit-backend/issues/73
 
 
 class TestPublishCommand(distutils.command.build_py.build_py):
@@ -164,6 +184,7 @@ setup(
     long_description=README_MD,
     long_description_content_type="text/markdown",
     cmdclass={
+        "clean": CleanCommandExtension,
         "test_publish": TestPublishCommand,
         "publish": PublishCommand,
         "start_test_relayer": StartTestRelayerCommand,
@@ -177,7 +198,7 @@ setup(
             "0x-contract-artifacts",
             "0x-contract-addresses",
             "0x-order-utils",
-            "0x-web3",
+            "web3",
             "bandit",
             "black",
             "coverage",

python-packages/sra_client/src/zero_ex/sra_client/__init__.py

@@ -76,7 +76,7 @@ What network is it?
 
 For our Maker role, we'll just use the first address available in the node:
 
->>> maker_address = Web3(eth_node).eth.accounts[0].lower()
+>>> maker_address = Web3(eth_node).eth.accounts[0]
 
 The 0x Ganache snapshot loaded into our eth_node has a pre-loaded ZRX balance
 for this account, so the example orders below have the maker trading away ZRX.
@@ -284,7 +284,7 @@ examples.
 Filling
 ^^^^^^^
 
->>> taker_address = Web3(eth_node).eth.accounts[1].lower()
+>>> taker_address = Web3(eth_node).eth.accounts[1]
 
 Our taker will take a ZRX/WETH order, but it doesn't have any WETH yet.  By
 depositing some ether into the WETH contract, it will be given some WETH to
@@ -304,8 +304,10 @@ Next the taker needs to give the 0x contracts permission to trade their WETH:
 
 >>> weth_instance.functions.approve(
 ...     Web3.toChecksumAddress(contract_addresses.erc20_proxy),
-...     1000000000000000000).transact(
+...     1000000000000000000
-...     {"from": Web3.toChecksumAddress(taker_address)})
+... ).transact(
+...     {"from": Web3.toChecksumAddress(taker_address)}
+... )
 HexBytes('0x...')
 
 Now the taker is ready to trade.
@@ -313,12 +315,19 @@ Now the taker is ready to trade.
 Recall that in a previous example we selected a specific order from the order
 book.  Now let's have the taker fill it:
 
->>> from zero_ex.contract_wrappers import Exchange, TxParams
+>>> from zero_ex.contract_wrappers import TxParams
+>>> from zero_ex.contract_wrappers.exchange import Exchange
 >>> from zero_ex.order_utils import Order
 >>> exchange = Exchange(
 ...     provider=eth_node,
 ...     contract_address=NETWORK_TO_ADDRESSES[NetworkId.GANACHE].exchange
 ... )
+
+(Due to `an Issue with the Launch Kit Backend
+<https://github.com/0xProject/0x-launch-kit-backend/issues/73>`_, we need to
+checksum the address in the order before filling it.)
+>>> order['makerAddress'] = Web3.toChecksumAddress(order['makerAddress'])
+
 >>> exchange.fill_order.send_transaction(
 ...     order=order,
 ...     taker_asset_fill_amount=order['makerAssetAmount']/2, # note the half fill

python-packages/sra_client/test/relayer/docker-compose.yml

@@ -6,7 +6,7 @@ services:
         ports:
             - "8545:8545"
     launchkit:
-        image: "0xorg/launch-kit-ci"
+        image: "0xorg/launch-kit-backend:74bcc39"
         depends_on:
             - ganache
         ports:
@@ -16,4 +16,5 @@ services:
             - NETWORK_ID=50
             - RPC_URL=http://localhost:8545
             - WHITELIST_ALL_TOKENS=True
-        command: bash -c "until curl -sfd'{\"method\":\"net_listening\"}' http://localhost:8545 | grep true; do continue; done; forever ts/lib/index.js"
+        command: |
+            sh -c "until printf 'POST /\r\nContent-Length: 26\r\n\r\n{\"method\":\"net_listening\"}' | nc localhost 8545 | grep true; do continue; done; node_modules/.bin/forever ts/lib/index.js"

python-packages/tsort

@@ -0,0 +1,17 @@
+#!/usr/bin/env bash
+
+# extract package interdependencies and topographically sort them
+
+find -name setup.py -exec grep -H \"0x- {} \; | \
+    grep -v name= | \
+    grep -v NAME | \
+    sed \
+        -e 's/^\.\//0x-/' \
+        -e 's/\/setup.py://' \
+        -e 's/"//g' -e 's/,$//' \
+        -e 's/_/-/g' | \
+    tsort | \
+    tac | \
+    sed \
+        -e 's/^0x-//' \
+        -e 's/-/_/'