213 Commits

Author SHA1 Message Date
Alex Svanevik
bca44bf9e3
Merge pull request #1684 from 0xProject/pull-github-data
Pull Github data
2019-03-22 18:33:40 +08:00
askeluv
08e49dcf2e Throttling to respect rate limits 2019-03-22 18:15:49 +08:00
Fabio Berger
0f45409b4d Publish
- 0x.js@6.0.5
 - @0x/abi-gen@2.0.8
 - @0x/abi-gen-wrappers@4.1.0
 - @0x/assert@2.0.8
 - @0x/asset-buyer@6.0.5
 - @0x/base-contract@5.0.4
 - @0x/connect@5.0.4
 - @0x/contract-addresses@2.3.0
 - @0x/contract-artifacts@1.4.0
 - @0x/contract-wrappers@8.0.5
 - @0x/contracts-gen@1.0.7
 - @0x/dev-tools-pages@0.0.24
 - @0x/dev-utils@2.2.0
 - @0x/fill-scenarios@3.0.4
 - @0x/instant@1.0.19
 - @0x/json-schemas@3.0.8
 - @0x/metacoin@0.0.46
 - @0x/migrations@4.1.0
 - @0x/monorepo-scripts@1.0.29
 - @0x/order-utils@7.1.1
 - @0x/order-watcher@4.0.5
 - @0x/pipeline@1.0.16
 - @0x/react-docs@2.0.9
 - @0x/react-shared@2.0.9
 - @0x/sol-compiler@3.1.5
 - @0x/sol-coverage@3.0.2
 - @0x/sol-doc@2.0.9
 - @0x/sol-profiler@3.1.4
 - @0x/sol-resolver@2.0.6
 - @0x/sol-trace@2.0.10
 - @0x/sol-tracing-utils@6.0.9
 - @0x/sra-spec@2.0.8
 - @0x/subproviders@4.0.4
 - @0x/testnet-faucets@1.0.75
 - @0x/types@2.2.1
 - @0x/typescript-typings@4.2.1
 - @0x/utils@4.3.0
 - @0x/web3-wrapper@6.0.4
 - @0x/website@0.0.78
 - @0x/contracts-asset-proxy@2.1.0
 - @0x/contracts-coordinator@1.1.0
 - @0x/contracts-erc1155@1.1.0
 - @0x/contracts-erc20@2.1.0
 - @0x/contracts-erc721@2.1.0
 - @0x/contracts-exchange@2.1.0
 - @0x/contracts-exchange-forwarder@2.1.0
 - @0x/contracts-exchange-libs@2.1.0
 - @0x/contracts-extensions@3.1.0
 - @0x/contracts-multisig@3.1.0
 - @0x/contracts-test-utils@3.1.1
 - @0x/contracts-utils@3.1.0
2019-03-21 17:07:20 +01:00
Leonid Logvinov
d4c37ecfa3
Publish
- 0x.js@6.0.4
 - @0x/abi-gen@2.0.7
 - @0x/abi-gen-wrappers@4.0.3
 - @0x/assert@2.0.7
 - @0x/asset-buyer@6.0.4
 - @0x/base-contract@5.0.3
 - @0x/connect@5.0.3
 - @0x/contract-addresses@2.2.3
 - @0x/contract-wrappers@8.0.4
 - @0x/contracts-gen@1.0.6
 - @0x/dev-tools-pages@0.0.23
 - @0x/dev-utils@2.1.4
 - ethereum-types@2.1.1
 - @0x/fill-scenarios@3.0.3
 - @0x/instant@1.0.18
 - @0x/json-schemas@3.0.7
 - @0x/metacoin@0.0.45
 - @0x/migrations@4.0.4
 - @0x/monorepo-scripts@1.0.28
 - @0x/order-utils@7.1.0
 - @0x/order-watcher@4.0.4
 - @0x/pipeline@1.0.15
 - @0x/react-docs@2.0.8
 - @0x/react-shared@2.0.8
 - @0x/sol-compiler@3.1.4
 - @0x/sol-coverage@3.0.1
 - @0x/sol-doc@2.0.8
 - @0x/sol-profiler@3.1.3
 - @0x/sol-resolver@2.0.5
 - @0x/sol-trace@2.0.9
 - @0x/sol-tracing-utils@6.0.8
 - @0x/sra-spec@2.0.7
 - @0x/subproviders@4.0.3
 - @0x/testnet-faucets@1.0.74
 - @0x/types@2.2.0
 - @0x/typescript-typings@4.2.0
 - @0x/utils@4.2.3
 - @0x/web3-wrapper@6.0.3
 - @0x/website@0.0.77
 - @0x/contracts-asset-proxy@2.0.0
 - @0x/contracts-coordinator@1.0.0
 - @0x/contracts-erc1155@1.0.1
 - @0x/contracts-erc20@2.0.0
 - @0x/contracts-erc721@2.0.0
 - @0x/contracts-exchange@2.0.0
 - @0x/contracts-exchange-forwarder@2.0.0
 - @0x/contracts-exchange-libs@2.0.0
 - @0x/contracts-extensions@3.0.0
 - @0x/contracts-multisig@3.0.0
 - @0x/contracts-test-utils@3.1.0
 - @0x/contracts-utils@3.0.0
2019-03-20 15:26:54 +01:00
askeluv
346c6fc590 Deal with failing Github API for issues 2019-03-20 12:41:56 +08:00
Alex Svanevik
3bdc1802cb
Merge branch 'development' into pull-github-data 2019-03-19 14:31:31 +08:00
Alex Svanevik
a8720806f1
Merge pull request #1702 from 0xProject/fix/pipeline/replace-edps-endpoint
Changed EDPS endpoint to a 0x AWS one
2019-03-18 18:09:31 +08:00
askeluv
5aac4c2e5d Linting 2019-03-18 17:36:19 +08:00
Xianny
7326dbd108
add greenhouse (#1690) 2019-03-15 10:37:52 -07:00
F. Eugene Aumson
4bf311a282
Enable cryptokitties data from nonfungible.com (#1680)
* enable cryptokitties

    Recall that nonfungible.com sent us an initial data dump so that we
    wouldn't have to pull everything from the API. And recall that we're
    storing that initial dump on S3. Finally, recall that the cryptokitties
    data was particularly voluminous (~1GB), which doesn't easily transfer to
    and from S3.

    These changes provide a CLI script to partition that data (or any such data
    from nonfungible.com) into a set of smaller files; and a scraping script
    that seamlessly recombines those partitions for loading into the database.

* add chainbreakers, chibifighters & mlbcryptobaseball
2019-03-15 10:50:59 -04:00
F. Eugene Aumson
7f5a3f12ca
Relay failures through command line interface (#1700)
* Relay failures through command line interface
2019-03-15 10:38:21 -04:00
askeluv
6a9b71466d Changed endpoint to an AWS one + added increased timeout 2019-03-15 15:52:32 +08:00
askeluv
095882d016 Added Github issue entity 2019-03-13 16:30:10 +08:00
askeluv
1693506f80 name -> full_name + fixed error message for missing access token 2019-03-13 11:47:42 +08:00
askeluv
b88e42a52d Prettier 2019-03-11 14:33:17 +08:00
askeluv
8d1b27d130 Added entity tests 2019-03-11 14:31:51 +08:00
askeluv
1148d37102 Added comparison details to forks vs main branch 2019-03-11 14:10:08 +08:00
askeluv
88704ce417 Prettier 2019-03-08 20:07:26 +08:00
askeluv
9f69d2cb76 Third entity: Github fork 2019-03-08 20:04:03 +08:00
askeluv
55fd71c5e1 Second entity: Github pull request 2019-03-08 14:37:36 +08:00
askeluv
1991bd437f First entity: Github repo 2019-03-08 12:32:21 +08:00
Alex Browne
a70931ffbf
Add documentation for some pipeline entities. 2019-03-07 18:28:57 -08:00
Leonid Logvinov
397496aff0
Publish
- 0x.js@6.0.3
 - @0x/abi-gen@2.0.6
 - @0x/abi-gen-wrappers@4.0.2
 - @0x/assert@2.0.6
 - @0x/asset-buyer@6.0.3
 - @0x/base-contract@5.0.2
 - @0x/connect@5.0.2
 - @0x/contract-wrappers@8.0.3
 - @0x/contracts-gen@1.0.5
 - @0x/dev-tools-pages@0.0.22
 - @0x/dev-utils@2.1.3
 - @0x/fill-scenarios@3.0.2
 - @0x/instant@1.0.17
 - @0x/json-schemas@3.0.6
 - @0x/metacoin@0.0.44
 - @0x/migrations@4.0.3
 - @0x/monorepo-scripts@1.0.27
 - @0x/order-utils@7.0.2
 - @0x/order-watcher@4.0.3
 - @0x/pipeline@1.0.14
 - @0x/react-docs@2.0.7
 - @0x/react-shared@2.0.7
 - @0x/sol-compiler@3.1.3
 - @0x/sol-coverage@3.0.0
 - @0x/sol-doc@2.0.7
 - @0x/sol-profiler@3.1.2
 - @0x/sol-trace@2.0.8
 - @0x/sol-tracing-utils@6.0.7
 - @0x/sra-spec@2.0.6
 - @0x/subproviders@4.0.2
 - @0x/testnet-faucets@1.0.73
 - @0x/utils@4.2.2
 - @0x/web3-wrapper@6.0.2
 - @0x/website@0.0.76
 - @0x/contracts-asset-proxy@1.0.9
 - @0x/contracts-erc20@1.0.9
 - @0x/contracts-erc721@1.0.9
 - @0x/contracts-exchange@1.0.9
 - @0x/contracts-exchange-forwarder@1.0.9
 - @0x/contracts-exchange-libs@1.1.3
 - @0x/contracts-extensions@2.0.8
 - @0x/contracts-multisig@2.0.8
 - @0x/contracts-tec@0.0.4
 - @0x/contracts-test-utils@3.0.8
 - @0x/contracts-utils@2.0.8
2019-03-01 14:34:26 -08:00
Alex Svanevik
99318ae2ba
Merge pull request #1658 from 0xProject/fix/pipeline/edps-rate-limiting
Added bottleneck for EDPS to deal with rate-limiting
2019-03-01 18:11:18 +08:00
askeluv
338cc69034 Removed logging 2019-03-01 14:33:19 +08:00
F. Eugene Aumson
3b5f0d5c30
Adapt to Bloxy API change (#1659)
* Adapt to Bloxy API change

We were using the `days` parameter, but that's no longer available in
the API, so our fetching of previous days (before "today") was broken.

This change gets rid of the concept of "days", and uses the API's new
`from_date` and `till_date` parameters to fetch previous days' trades.

The change to the query in `getLastSeenTimestampAsync()` was necessary
because it was returning a string (despite the `as Array<{ tx_timestamp:
number }>` cast), which was later causing problems trying to pass that
value into `new Date()`.  It worked before because we were doing some
math operations on it (multiplying it by some numbers), so coercion was
saving us.  With the change from a raw query to a `typeorm` call, the
`numberToBigIntTransformer` specified in the `DexTrade` entity is now
doing the proper type conversion for us.

The new `MAX_DAYS` constant in `pull_competing_dex_trades.ts` is
necessary in order to avoid trying to pull data all the way back to 1969
:D, which induces the API to return an HTTP 503.

* Increase trades per query to match current API doc

This is needed in order to be able to pull more than a handful of days
of data at a time (at least, with the script the way its currently
written).
2019-02-28 20:28:00 -05:00
askeluv
e450191548 Added bottleneck to deal with rate-limiting 2019-02-28 13:08:09 +08:00
Francesco Agosti
3aee83f3d8
Merge pull request #1633 from 0xProject/feature/pipeline/concepts-schema-and-radar-depth-table
[pipeline] Add concepts schema and radar orderbook USD price concept
2019-02-27 20:40:00 -08:00
fragosti
3610a2bc8d Run linter 2019-02-27 20:23:10 -08:00
fragosti
ab559d4620 Include new test path in package.json 2019-02-27 20:11:48 -08:00
fragosti
15d308d4c5 Remove isCli method 2019-02-27 19:34:35 -08:00
Alex Svanevik
e7ea66afb5
Merge pull request #1629 from 0xProject/etherscan-exchange-transactions
Pull exchange contract transactions from Etherscan
2019-02-28 08:30:05 +08:00
Leonid Logvinov
e81ae05df9
Publish
- 0x.js@6.0.2
 - @0x/asset-buyer@6.0.2
 - @0x/contract-wrappers@8.0.2
 - @0x/instant@1.0.16
 - @0x/metacoin@0.0.43
 - @0x/migrations@4.0.2
 - @0x/order-watcher@4.0.2
 - @0x/pipeline@1.0.13
 - @0x/sol-compiler@3.1.2
 - @0x/sol-coverage@2.0.6
 - @0x/sol-doc@2.0.6
 - @0x/sol-profiler@3.1.1
 - @0x/sol-trace@2.0.7
 - @0x/sol-tracing-utils@6.0.6
 - @0x/testnet-faucets@1.0.72
 - @0x/website@0.0.75
 - @0x/contracts-asset-proxy@1.0.8
 - @0x/contracts-erc20@1.0.8
 - @0x/contracts-erc721@1.0.8
 - @0x/contracts-exchange@1.0.8
 - @0x/contracts-exchange-forwarder@1.0.8
 - @0x/contracts-exchange-libs@1.1.2
 - @0x/contracts-extensions@2.0.7
 - @0x/contracts-multisig@2.0.7
 - @0x/contracts-tec@0.0.3
 - @0x/contracts-test-utils@3.0.7
 - @0x/contracts-utils@2.0.7
2019-02-27 12:45:19 -08:00
fragosti
42c3bb00ec move to using require.main === module 2019-02-26 16:05:20 -08:00
Leonid Logvinov
ec24c79da1
Publish
- 0x.js@6.0.1
 - @0x/abi-gen@2.0.5
 - @0x/abi-gen-wrappers@4.0.1
 - @0x/assert@2.0.5
 - @0x/asset-buyer@6.0.1
 - @0x/base-contract@5.0.1
 - @0x/connect@5.0.1
 - @0x/contract-wrappers@8.0.1
 - @0x/contracts-gen@1.0.4
 - @0x/dev-tools-pages@0.0.21
 - @0x/dev-utils@2.1.2
 - @0x/fill-scenarios@3.0.1
 - @0x/instant@1.0.15
 - @0x/json-schemas@3.0.5
 - @0x/metacoin@0.0.42
 - @0x/migrations@4.0.1
 - @0x/monorepo-scripts@1.0.26
 - @0x/order-utils@7.0.1
 - @0x/order-watcher@4.0.1
 - @0x/pipeline@1.0.12
 - @0x/react-docs@2.0.6
 - @0x/react-shared@2.0.6
 - @0x/sol-compiler@3.1.1
 - @0x/sol-coverage@2.0.5
 - @0x/sol-doc@2.0.5
 - @0x/sol-profiler@3.1.0
 - @0x/sol-resolver@2.0.4
 - @0x/sol-trace@2.0.6
 - @0x/sol-tracing-utils@6.0.5
 - @0x/sra-spec@2.0.5
 - @0x/subproviders@4.0.1
 - @0x/testnet-faucets@1.0.71
 - @0x/types@2.1.1
 - @0x/utils@4.2.1
 - @0x/web3-wrapper@6.0.1
 - @0x/website@0.0.74
 - @0x/contracts-asset-proxy@1.0.7
 - @0x/contracts-erc20@1.0.7
 - @0x/contracts-erc721@1.0.7
 - @0x/contracts-exchange@1.0.7
 - @0x/contracts-exchange-forwarder@1.0.7
 - @0x/contracts-exchange-libs@1.1.1
 - @0x/contracts-extensions@2.0.6
 - @0x/contracts-multisig@2.0.6
 - @0x/contracts-tec@0.0.2
 - @0x/contracts-test-utils@3.0.6
 - @0x/contracts-utils@2.0.6
2019-02-26 14:45:52 -08:00
fragosti
eb212de70e run linter and add doc string 2019-02-26 14:05:18 -08:00
fragosti
8fbdef2a1d Make job look at last timestamp, fix tests 2019-02-26 13:52:18 -08:00
askeluv
85ed5d27f5 Bumped up timeout to 4 mins 2019-02-26 19:37:59 +08:00
askeluv
6ac9e11245 Linting 2019-02-26 11:46:21 +08:00
askeluv
9fbd809344 Improved testing - added fixtures 2019-02-26 11:42:53 +08:00
Leonid Logvinov
5934eb0559
Publish
- 0x.js@6.0.0
 - @0x/abi-gen@2.0.4
 - @0x/abi-gen-templates@2.0.1
 - @0x/abi-gen-wrappers@4.0.0
 - @0x/assert@2.0.4
 - @0x/asset-buyer@6.0.0
 - @0x/base-contract@5.0.0
 - @0x/connect@5.0.0
 - @0x/contract-addresses@2.2.2
 - @0x/contract-wrappers@8.0.0
 - @0x/contracts-gen@1.0.3
 - @0x/dev-tools-pages@0.0.20
 - @0x/dev-utils@2.1.1
 - ethereum-types@2.1.0
 - @0x/fill-scenarios@3.0.0
 - @0x/instant@1.0.14
 - @0x/json-schemas@3.0.4
 - @0x/metacoin@0.0.41
 - @0x/migrations@4.0.0
 - @0x/monorepo-scripts@1.0.25
 - @0x/order-utils@7.0.0
 - @0x/order-watcher@4.0.0
 - @0x/pipeline@1.0.11
 - @0x/react-docs@2.0.5
 - @0x/react-shared@2.0.5
 - @0x/sol-compiler@3.1.0
 - @0x/sol-coverage@2.0.4
 - @0x/sol-doc@2.0.4
 - @0x/sol-profiler@3.0.0
 - @0x/sol-resolver@2.0.3
 - @0x/sol-trace@2.0.5
 - @0x/sol-tracing-utils@6.0.4
 - @0x/sra-spec@2.0.4
 - @0x/subproviders@4.0.0
 - @0x/testnet-faucets@1.0.70
 - @0x/types@2.1.0
 - @0x/typescript-typings@4.1.0
 - @0x/utils@4.2.0
 - @0x/web3-wrapper@6.0.0
 - @0x/website@0.0.73
 - @0x/contracts-asset-proxy@1.0.6
 - @0x/contracts-erc20@1.0.6
 - @0x/contracts-erc721@1.0.6
 - @0x/contracts-exchange@1.0.6
 - @0x/contracts-exchange-forwarder@1.0.6
 - @0x/contracts-exchange-libs@1.1.0
 - @0x/contracts-extensions@2.0.5
 - @0x/contracts-multisig@2.0.5
 - @0x/contracts-test-utils@3.0.5
 - @0x/contracts-utils@2.0.5
2019-02-25 13:35:29 -08:00
fragosti
3191de68b8 Fix typo in query 2019-02-22 16:13:48 -08:00
fragosti
f3da56773e Run prettier 2019-02-22 12:02:58 -08:00
fragosti
5197758579 Add missing file 2019-02-22 10:36:30 -08:00
askeluv
6eb923d22f Throw error when API key is missing + use response in parser test 2019-02-22 11:21:33 +08:00
fragosti
e602afcd5f add documentation 2019-02-21 18:24:11 -08:00
askeluv
bf1115d417 Backfills automatically + prettier 2019-02-22 09:45:25 +08:00
fragosti
f5d668af31 Add radar_orderbook_usd_prices concept 2019-02-21 15:45:36 -08:00
fragosti
62373f969c Add concepts migrations 2019-02-21 12:51:27 -08:00
askeluv
8885f543ae Pull exchange contract transactions from Etherscan 2019-02-21 20:37:55 +08:00