Commit Graph

19 Commits

Author SHA1 Message Date
askeluv
8885f543ae Pull exchange contract transactions from Etherscan 2019-02-21 20:37:55 +08:00
Francesco Agosti
39390865da Merge pull request #1612 from 0xProject/feature/pipeline/add-radar
[pipeline] Add Radar orders and maker_address column to token_orderbook_snapshots
2019-02-20 14:17:42 -08:00
Alex Svanevik
154e7a6a8a Merge pull request #1601 from askeluv/feature/pipeline/edps-slippage
Slippage calculations from Ethereum DEX Prices Service (EDPS)
2019-02-20 22:51:23 +08:00
F. Eugene Aumson
1232a9a03d Integrate one-time dump and API for nonfungible.com (#1603)
* Add script for pulling NFT trade data from nonfungible.com

* corrections for current state of API

* change data model to match data source

* change primary key

* pull data from initial dump first, then API

* pull all supported NFT's, not just cryptokitties

* disable problematic data sources

* rename function to satisfy linter

* Rename table to nonfungible_dot_com_trades

* rename parser module to nonfungible_dot_com from non_fungible_dot_com, for consistency

* correct mistaken reference to Bloxy

* rename NonfungibleDotComTrade to ...TradeResponse

* `NftTrade` -> `NonfungibleDotComTrade`

* rename files to match prior object renaming

* use fetchAsync instead of axios

* improve fetchAsync error message: include URL

* avoid non-null contraints in API trades too, not just for trades from the one-time dump

* disable mythereum publisher
2019-02-19 19:07:42 -08:00
fragosti
87da6a947d Have migration to maker_address column working 2019-02-14 09:22:50 -08:00
fragosti
d1cc08f1d9 Attempt to add maker_address token_orderbook_snapshots 2019-02-13 11:03:40 -08:00
askeluv
82dffe9d0e Added remaining tests + fixed linting issues 2019-02-12 19:34:05 +08:00
askeluv
d7825dd7db Added test for entity 2019-02-12 16:06:53 +08:00
askeluv
c0a2f429b6 Added saving to database 2019-02-12 12:30:10 +08:00
askeluv
7ad4cb0078 [WIP] Slippage calculations from Ethereum DEX Prices Service (EDPS) 2019-02-08 18:54:38 +08:00
fragosti
c2ad95fd94 update types to adhere to new dex trades and bloxy api format 2019-01-30 10:12:15 -08:00
Xianny
27fc640a9e fetch and save copper (#1472)
Fetch and save Copper CRM
2019-01-08 13:50:51 -08:00
Alex Browne
b4cdb14b9b Refactor event scraping and add support for scraping ERC20 approval events (#1401)
* Refactor event scraping and add support for scraping ERC20 approval events

* Add tests for data_sources/contract-wrappers/utils
2018-12-11 15:16:05 -08:00
Alex Browne
e0348f9c04 Change type of transactions.gas_used and gas_price to BigNumber/numeric 2018-12-04 20:20:49 -08:00
Alex Browne
00f86ca0f7 Address PR feedback 2018-12-04 20:04:08 -08:00
Xianny
8c21a700ba pull OHLCV records from Crypto Compare (#1349)
* [WIP] pull OHLCV records from Crypto Compare

* lint

* refactor to pull logic out of script and into modules

* add entity test for ohlcv_external entity

* implement rate limit and chronological backfill for ohlcv

* add unit tests; cleanup variable names

* Fetch OHLCV pairs params from events table

* better method names

* fix outdated test

* lint

* Clean up after review

* oops

* fix failing test

* better filtering of most recent records

* fix bug when generating pairs

* fix default earliest backfill date

* fix bug with retrieving backfill time

* prettier
2018-12-04 14:26:03 -08:00
zkao
87ffa5d7ab Token_orderbook_snapshots for Ddex and Paradex(#1354)
* Implements the TokenOrderbookSnapshot Table

* Scripts, Data Sources and Entities to pull Ddex and Paradex API data.
2018-12-04 14:25:42 -08:00
Alex Browne
7198b441e0 Add script for parsing competing dex trades from Bloxy (#1355) 2018-12-04 14:25:42 -08:00
Alex Browne
3d211c415b Introduce framework for running basic tests for entities (#1344)
* Introduce framework for running basic tests for entities

* Add pipeline tests to CircleCI config

* Make pipeline tests more configurable and fix CircleCI config

* Add coverage dir to pipeline package

* Add basic tests for all exchange event entities

* Add tests for remaining entities

* Create separate test scripts in package.json and add new info to README

* Update db_setup.ts to revert migrations even if you are using docker

* Automatically pull the postgres image if needed

* Add comment about why NumberToBigIntTransformer is needed
2018-12-04 14:25:38 -08:00