* enable cryptokitties
Recall that nonfungible.com sent us an initial data dump so that we
wouldn't have to pull everything from the API. And recall that we're
storing that initial dump on S3. Finally, recall that the cryptokitties
data was particularly voluminous (~1GB), which doesn't easily transfer to
and from S3.
These changes provide a CLI script to partition that data (or any such data
from nonfungible.com) into a set of smaller files; and a scraping script
that seamlessly recombines those partitions for loading into the database.
* add chainbreakers, chibifighters & mlbcryptobaseball
* Adapt to Bloxy API change
We were using the `days` parameter, but that's no longer available in
the API, so our fetching of previous days (before "today") was broken.
This change gets rid of the concept of "days", and uses the API's new
`from_date` and `till_date` parameters to fetch previous days' trades.
The change to the query in `getLastSeenTimestampAsync()` was necessary
because it was returning a string (despite the `as Array<{ tx_timestamp:
number }>` cast), which was later causing problems trying to pass that
value into `new Date()`. It worked before because we were doing some
math operations on it (multiplying it by some numbers), so coercion was
saving us. With the change from a raw query to a `typeorm` call, the
`numberToBigIntTransformer` specified in the `DexTrade` entity is now
doing the proper type conversion for us.
The new `MAX_DAYS` constant in `pull_competing_dex_trades.ts` is
necessary in order to avoid trying to pull data all the way back to 1969
:D, which induces the API to return an HTTP 503.
* Increase trades per query to match current API doc
This is needed in order to be able to pull more than a handful of days
of data at a time (at least, with the script the way its currently
written).
* Add script for pulling NFT trade data from nonfungible.com
* corrections for current state of API
* change data model to match data source
* change primary key
* pull data from initial dump first, then API
* pull all supported NFT's, not just cryptokitties
* disable problematic data sources
* rename function to satisfy linter
* Rename table to nonfungible_dot_com_trades
* rename parser module to nonfungible_dot_com from non_fungible_dot_com, for consistency
* correct mistaken reference to Bloxy
* rename NonfungibleDotComTrade to ...TradeResponse
* `NftTrade` -> `NonfungibleDotComTrade`
* rename files to match prior object renaming
* use fetchAsync instead of axios
* improve fetchAsync error message: include URL
* avoid non-null contraints in API trades too, not just for trades from the one-time dump
* disable mythereum publisher
