92 Commits

Author SHA1 Message Date
zkao
6d45beccad
Fix dex order quote/base asset assigning (#1432) 2018-12-14 12:15:35 -08:00
Leonid Logvinov
2abd8fe4ee
Publish
- 0x.js@2.0.8
 - @0x/abi-gen@1.0.19
 - @0x/abi-gen-wrappers@2.0.2
 - @0x/assert@1.0.20
 - @0x/asset-buyer@3.0.4
 - @0x/base-contract@3.0.10
 - @0x/connect@3.0.10
 - @0x/contract-wrappers@4.1.3
 - @0x/dev-tools-pages@0.0.10
 - @0x/dev-utils@1.0.21
 - ethereum-types@1.1.4
 - @0x/fill-scenarios@1.0.16
 - @0x/instant@1.0.4
 - @0x/json-schemas@2.1.4
 - @0x/metacoin@0.0.32
 - @0x/migrations@2.2.2
 - @0x/order-utils@3.0.7
 - @0x/order-watcher@2.2.8
 - @0x/pipeline@1.0.2
 - @0x/react-docs@1.0.22
 - @0x/react-shared@1.0.25
 - @0x/sol-compiler@1.1.16
 - @0x/sol-cov@2.1.16
 - @0x/sol-doc@1.0.11
 - @0x/sol-resolver@1.1.1
 - @0x/sra-spec@1.0.13
 - @0x/subproviders@2.1.8
 - @0x/testnet-faucets@1.0.60
 - @0x/tslint-config@2.0.0
 - @0x/types@1.4.1
 - @0x/typescript-typings@3.0.6
 - @0x/utils@2.0.8
 - @0x/web3-wrapper@3.2.1
 - @0x/website@0.0.63
 - @0x/contracts-examples@1.0.1
 - @0x/contracts-extensions@1.0.1
 - @0x/contracts-interfaces@1.0.1
 - @0x/contracts-libs@1.0.1
 - @0x/contracts-multisig@1.0.1
 - @0x/contracts-protocol@2.1.58
 - @0x/contracts-test-utils@1.0.2
 - @0x/contracts-tokens@1.0.1
 - @0x/contracts-utils@1.0.1
2018-12-13 14:28:27 -08:00
Xianny
8906c30f37
add special case to scrape OHLCV for eth/usd (#1428) 2018-12-13 13:13:28 -08:00
Alex Browne
3a8f3e01b6
Fix bug that occurs when token metadata decimals is undefined. (#1426)
* Fix bug that occurs when token metadata decimals is undefined.

* move toBigNumberOrNull to utils
2018-12-12 16:49:01 -08:00
Leonid Logvinov
1534505652
Fix linter errors 2018-12-12 15:32:41 -08:00
Alex Browne
5cff911035
Make pull_missing_blocks script consider all events with block numbers (#1420) 2018-12-11 17:31:16 -08:00
zkao
42be1a429f
track idex orderbook snapshots (#1397)
* Track Idex and Oasis Orderbook Snapshots
2018-12-11 15:48:54 -08:00
Fabio Berger
96b8100a78 Publish
- 0x.js@2.0.7
 - @0x/abi-gen@1.0.18
 - @0x/abi-gen-wrappers@2.0.1
 - @0x/assert@1.0.19
 - @0x/asset-buyer@3.0.3
 - @0x/base-contract@3.0.9
 - @0x/connect@3.0.9
 - @0x/contract-wrappers@4.1.2
 - @0x/dev-tools-pages@0.0.9
 - @0x/dev-utils@1.0.20
 - ethereum-types@1.1.3
 - @0x/fill-scenarios@1.0.15
 - @0x/instant@1.0.3
 - @0x/json-schemas@2.1.3
 - @0x/metacoin@0.0.31
 - @0x/migrations@2.2.1
 - @0x/monorepo-scripts@1.0.15
 - @0x/order-utils@3.0.6
 - @0x/order-watcher@2.2.7
 - @0x/pipeline@1.0.1
 - @0x/react-docs@1.0.21
 - @0x/react-shared@1.0.24
 - @0x/sol-compiler@1.1.15
 - @0x/sol-cov@2.1.15
 - @0x/sol-doc@1.0.10
 - @0x/sol-resolver@1.1.0
 - @0x/sra-spec@1.0.12
 - @0x/subproviders@2.1.7
 - @0x/testnet-faucets@1.0.59
 - @0x/types@1.4.0
 - @0x/typescript-typings@3.0.5
 - @0x/utils@2.0.7
 - @0x/web3-wrapper@3.2.0
 - @0x/website@0.0.62
 - @0x/contracts-examples@1.0.1
 - @0x/contracts-extensions@1.0.1
 - @0x/contracts-interfaces@1.0.1
 - @0x/contracts-libs@1.0.1
 - @0x/contracts-multisig@1.0.1
 - @0x/contracts-protocol@2.1.57
 - @0x/contracts-test-utils@1.0.1
 - @0x/contracts-tokens@1.0.1
 - @0x/contracts-utils@1.0.1
2018-12-11 15:38:43 -08:00
Alex Browne
b4cdb14b9b
Refactor event scraping and add support for scraping ERC20 approval events (#1401)
* Refactor event scraping and add support for scraping ERC20 approval events

* Add tests for data_sources/contract-wrappers/utils
2018-12-11 15:16:05 -08:00
Xianny
7be2dab9d5
perform null check on token metadata (#1412) 2018-12-10 15:08:13 -08:00
Xianny
02e14f88d2
Fix/pipeline/ohlcv ratelimit (#1403)
Use time-based throttling and increase batch size of CC query
2018-12-10 11:01:35 -08:00
xianny
096c4c8f2b change to camelCase 2018-12-10 10:57:36 -08:00
kao
f14603ca4d Implement prefer-template tslint rule 2018-12-07 17:06:52 -08:00
xianny
5febb595e9 lint: remove unused variable 2018-12-07 10:03:17 -08:00
xianny
2bea70a0a6 refresh rate limit every second instead of every minute 2018-12-07 09:53:49 -08:00
xianny
41f90c697b cleanup: stray import, rename variable 2018-12-07 09:48:26 -08:00
xianny
7b1471ffe8 query CC with larger batch size 2018-12-07 09:43:55 -08:00
xianny
8b3b4d983f rename variable and define default in only 1 location 2018-12-07 09:43:55 -08:00
xianny
ac971685b3 upgrade throttling code 2018-12-07 09:43:55 -08:00
Xianny
78d0ab1aa2
Fix/pipeline/ohlcv (#1393)
The OHLCV script in data pipeline quits early when we get no data from Crypto Compare. Sometimes Crypto Compare gives us a valid empty response (e.g. when we query for way back in time) and we need to just continue. This adds better filtering for the types of Crypto Compare responses to detect when we should continue and when we should really quit.
2018-12-05 16:05:06 -08:00
Alex Browne
08eb0b91b6 Fix RadarRelay timestamps (#1391)
* Fixing rr timestamps

* Apply prettier
2018-12-05 12:27:32 -08:00
Alex Browne
e0348f9c04
Change type of transactions.gas_used and gas_price to BigNumber/numeric 2018-12-04 20:20:49 -08:00
Alex Browne
2e704ac01a
Fix prettier 2018-12-04 20:08:32 -08:00
Alex Browne
00f86ca0f7
Address PR feedback 2018-12-04 20:04:08 -08:00
Fabio B
549f5e4655
Use a string template in packages/pipeline/src/scripts/pull_missing_events.ts
Co-Authored-By: albrow <stephenalexbrowne@gmail.com>
2018-12-04 19:56:23 -08:00
Fabio B
ebdc539da7
Fix typo in packages/pipeline/src/parsers/bloxy/index.ts
Co-Authored-By: albrow <stephenalexbrowne@gmail.com>
2018-12-04 19:31:19 -08:00
Alex Browne
8721d4ed7a
Fix linter 2018-12-04 15:23:15 -08:00
Xianny
8c21a700ba
pull OHLCV records from Crypto Compare (#1349)
* [WIP] pull OHLCV records from Crypto Compare

* lint

* refactor to pull logic out of script and into modules

* add entity test for ohlcv_external entity

* implement rate limit and chronological backfill for ohlcv

* add unit tests; cleanup variable names

* Fetch OHLCV pairs params from events table

* better method names

* fix outdated test

* lint

* Clean up after review

* oops

* fix failing test

* better filtering of most recent records

* fix bug when generating pairs

* fix default earliest backfill date

* fix bug with retrieving backfill time

* prettier
2018-12-04 14:26:03 -08:00
zkao
87ffa5d7ab
Token_orderbook_snapshots for Ddex and Paradex(#1354)
* Implements the TokenOrderbookSnapshot Table

* Scripts, Data Sources and Entities to pull Ddex and Paradex API data.
2018-12-04 14:25:42 -08:00
Alex Browne
7198b441e0
Add script for parsing competing dex trades from Bloxy (#1355) 2018-12-04 14:25:42 -08:00
Alex Browne
3d211c415b
Introduce framework for running basic tests for entities (#1344)
* Introduce framework for running basic tests for entities

* Add pipeline tests to CircleCI config

* Make pipeline tests more configurable and fix CircleCI config

* Add coverage dir to pipeline package

* Add basic tests for all exchange event entities

* Add tests for remaining entities

* Create separate test scripts in package.json and add new info to README

* Update db_setup.ts to revert migrations even if you are using docker

* Automatically pull the postgres image if needed

* Add comment about why NumberToBigIntTransformer is needed
2018-12-04 14:25:38 -08:00
Alex Browne
4061731245
[pipeline] Add additional documentation to the README (#1328) 2018-12-04 14:24:48 -08:00
Alex Browne
96fdb9b766
Reduce number of blocks per query in pull_missing_events 2018-12-04 14:24:48 -08:00
Jake Ellowitz
1aa3f9d69f
updating comment for 0x trusted tokens 2018-12-04 14:24:48 -08:00
Jake Ellowitz
dea89c4e22
metadata and trusted sources in same raw table 2018-12-04 14:24:48 -08:00
Jake Ellowitz
c6af5131b0
Pull token metadata re trusted tokens 2018-12-04 14:24:48 -08:00
Alex Browne
9986717671
Add script for pulling missing block data 2018-12-04 14:24:48 -08:00
Alex Browne
5cad2ad174
Check for special characters in table name in pull_missing_events 2018-12-04 14:24:48 -08:00
Alex Browne
24fd2d9730
Add support for pulling Cancel and CancelUpTo events 2018-12-04 14:24:48 -08:00
Alex Browne
0397ff8b22
Add transactionHash to CancelUpToEvent and CancelEvent entities 2018-12-04 14:24:48 -08:00
Alex Browne
1a25862086
Optimize database operations in pull_missing_events script 2018-12-04 14:24:48 -08:00
Alex Browne
b771020f88
Add homepage_url to relayers 2018-12-04 14:24:48 -08:00
Alex Browne
b42cf0c797
Add workaround for broken save method 2018-12-04 14:24:48 -08:00
Alex Browne
8076333795
Fix chunk size in pull_missing_events 2018-12-04 14:24:48 -08:00
Alex Browne
b0a2c10e11
Use built-in chunk feature of TypeORM save method 2018-12-04 14:24:48 -08:00
Alex Browne
303bbc42f4
Change some column types from varchar to numeric 2018-12-04 14:24:48 -08:00
Alex Browne
10e93bb01f
Add raw schema prefix to query in pull_missing_events 2018-12-04 14:24:48 -08:00
Alex Browne
a948305e7a
Rename taker and maker amounts in sra_orders 2018-12-04 14:24:48 -08:00
Alex Browne
26280e4aba
Implement scraping sra orders from radar relay 2018-12-04 14:24:48 -08:00
Alex Browne
55bbe1954b
Preliminary work for adding RR order book scraping 2018-12-04 14:24:48 -08:00