Commit Graph

36 Commits

Author SHA1 Message Date
Alex Svanevik
3bdc1802cb Merge branch 'development' into pull-github-data 2019-03-19 14:31:31 +08:00
Xianny
7326dbd108 add greenhouse (#1690) 2019-03-15 10:37:52 -07:00
askeluv
095882d016 Added Github issue entity 2019-03-13 16:30:10 +08:00
askeluv
8d1b27d130 Added entity tests 2019-03-11 14:31:51 +08:00
askeluv
1148d37102 Added comparison details to forks vs main branch 2019-03-11 14:10:08 +08:00
askeluv
9f69d2cb76 Third entity: Github fork 2019-03-08 20:04:03 +08:00
askeluv
55fd71c5e1 Second entity: Github pull request 2019-03-08 14:37:36 +08:00
askeluv
1991bd437f First entity: Github repo 2019-03-08 12:32:21 +08:00
askeluv
6ac9e11245 Linting 2019-02-26 11:46:21 +08:00
askeluv
9fbd809344 Improved testing - added fixtures 2019-02-26 11:42:53 +08:00
askeluv
6eb923d22f Throw error when API key is missing + use response in parser test 2019-02-22 11:21:33 +08:00
askeluv
bf1115d417 Backfills automatically + prettier 2019-02-22 09:45:25 +08:00
askeluv
8885f543ae Pull exchange contract transactions from Etherscan 2019-02-21 20:37:55 +08:00
Francesco Agosti
39390865da Merge pull request #1612 from 0xProject/feature/pipeline/add-radar
[pipeline] Add Radar orders and maker_address column to token_orderbook_snapshots
2019-02-20 14:17:42 -08:00
fragosti
4952cd5afb Factor out redundant source param in parse order functions 2019-02-20 13:45:14 -08:00
F. Eugene Aumson
1232a9a03d Integrate one-time dump and API for nonfungible.com (#1603)
* Add script for pulling NFT trade data from nonfungible.com

* corrections for current state of API

* change data model to match data source

* change primary key

* pull data from initial dump first, then API

* pull all supported NFT's, not just cryptokitties

* disable problematic data sources

* rename function to satisfy linter

* Rename table to nonfungible_dot_com_trades

* rename parser module to nonfungible_dot_com from non_fungible_dot_com, for consistency

* correct mistaken reference to Bloxy

* rename NonfungibleDotComTrade to ...TradeResponse

* `NftTrade` -> `NonfungibleDotComTrade`

* rename files to match prior object renaming

* use fetchAsync instead of axios

* improve fetchAsync error message: include URL

* avoid non-null contraints in API trades too, not just for trades from the one-time dump

* disable mythereum publisher
2019-02-19 19:07:42 -08:00
fragosti
12fd9c29f0 Add radar_orders parseRadarOrder test 2019-02-19 17:19:12 -08:00
fragosti
0c871b67ca Add parser test for Radar orderbook parsing 2019-02-19 16:58:11 -08:00
fragosti
c2ad95fd94 update types to adhere to new dex trades and bloxy api format 2019-01-30 10:12:15 -08:00
Xianny
a5b7a35160 upgrade to ddex api v3 (#1507) 2019-01-10 13:07:52 -08:00
Xianny
27fc640a9e fetch and save copper (#1472)
Fetch and save Copper CRM
2019-01-08 13:50:51 -08:00
Amir Bandeali
fd37e88bdb Use enums in pipeline 2019-01-07 11:11:19 -08:00
zkao
6d45beccad Fix dex order quote/base asset assigning (#1432) 2018-12-14 12:15:35 -08:00
zkao
42be1a429f track idex orderbook snapshots (#1397)
* Track Idex and Oasis Orderbook Snapshots
2018-12-11 15:48:54 -08:00
Alex Browne
b4cdb14b9b Refactor event scraping and add support for scraping ERC20 approval events (#1401)
* Refactor event scraping and add support for scraping ERC20 approval events

* Add tests for data_sources/contract-wrappers/utils
2018-12-11 15:16:05 -08:00
Xianny
8c21a700ba pull OHLCV records from Crypto Compare (#1349)
* [WIP] pull OHLCV records from Crypto Compare

* lint

* refactor to pull logic out of script and into modules

* add entity test for ohlcv_external entity

* implement rate limit and chronological backfill for ohlcv

* add unit tests; cleanup variable names

* Fetch OHLCV pairs params from events table

* better method names

* fix outdated test

* lint

* Clean up after review

* oops

* fix failing test

* better filtering of most recent records

* fix bug when generating pairs

* fix default earliest backfill date

* fix bug with retrieving backfill time

* prettier
2018-12-04 14:26:03 -08:00
zkao
87ffa5d7ab Token_orderbook_snapshots for Ddex and Paradex(#1354)
* Implements the TokenOrderbookSnapshot Table

* Scripts, Data Sources and Entities to pull Ddex and Paradex API data.
2018-12-04 14:25:42 -08:00
Alex Browne
7198b441e0 Add script for parsing competing dex trades from Bloxy (#1355) 2018-12-04 14:25:42 -08:00
Alex Browne
24fd2d9730 Add support for pulling Cancel and CancelUpTo events 2018-12-04 14:24:48 -08:00
Alex Browne
303bbc42f4 Change some column types from varchar to numeric 2018-12-04 14:24:48 -08:00
Alex Browne
26280e4aba Implement scraping sra orders from radar relay 2018-12-04 14:24:48 -08:00
Alex Browne
53cc9e9bed Rename table and column names 2018-12-04 14:24:48 -08:00
Alex Browne
ccad046eb6 Reorganize entities. Make scripts work from any directory. 2018-12-04 14:24:48 -08:00
Alex Browne
3a19faa5ff Update script to work with existing v1 pipeline data 2018-12-04 14:24:48 -08:00
Alex Browne
bbe1a843ef Update pipeline to use new @0x npm org for imports 2018-12-04 14:24:48 -08:00
Alex Browne
91e7485ecc Update to use ContractWrappers + Infura instead of Etherscan 2018-12-04 14:24:44 -08:00