Pair Programming #8: Pandas + NFT + Beeple’s 5,000 everydays

Youtube live link

Our goal today is to query OpenSea.io’s API for information about Beeple’s 5,000 day digital artwork NFT with Requests + Pandas.

This is boring on the data manipulation part, but super interesting on the application.

OpenSea.io has great documentation on how to query different assets and contracts.

In the video above and code below we will walk through how to query this API to get information on orders for Beeple’s 5,000 day piece that sold for $69M.

Pandas + Querying Non Fungible Token History

Let's query Beeple's 5,000 days on OpenSea.io. They have nice API documentation here https://docs.opensea.io/reference#retrieving-a-single-asset

Beeple: https://www.beeple-crap.com/everydays

In [1]:
import pandas as pd
import matplotlib.pyplot as plt
import requests

First we'll make our requests to get the raw response from the OpenSea.io api. I got this URL from the documentation above.

In [2]:
r = requests.get("https://api.opensea.io/api/v1/asset/0x2a46f2ffd99e19a89476e2f62270e0a35bbf0756/40913/")

Then let's view the JSON that comes out of the response. I'm going to limit the characters so it doesn't overload the page. Check out all of the information listed.

If you want to view the raw json in a nice form, head over to http://jsonviewer.stack.hu/ and paste in your json text

In [19]:
r.text[:500]
Out[19]:
'{"id":17984845,"token_id":"40913","num_sales":0,"background_color":null,"image_url":"https://lh3.googleusercontent.com/1gqAWnic2dGMSVC2mcHCWTK2aIfYBtKS5GFpsNryT6Gtxhj6_H_x7a14AnfA__nn_TWvI1Ankv90mj49JZa0G7QUkafOv4Tb31Z_8ZQ","image_preview_url":"https://lh3.googleusercontent.com/1gqAWnic2dGMSVC2mcHCWTK2aIfYBtKS5GFpsNryT6Gtxhj6_H_x7a14AnfA__nn_TWvI1Ankv90mj49JZa0G7QUkafOv4Tb31Z_8ZQ=s250","image_thumbnail_url":"https://lh3.googleusercontent.com/1gqAWnic2dGMSVC2mcHCWTK2aIfYBtKS5GFpsNryT6Gtxhj6_H_x7a'

Now I'm going to start the clean up process. Here I dig into the 'orders' field on the response and start to parse fields.

In [10]:
df_orders = pd.DataFrame(r.json()['orders'])
df_orders.head()
Out[10]:
created_dateclosing_dateclosing_extendableexpiration_timelisting_timeorder_hashmetadataexchangemakertaker...quantitysaltvrsapproved_on_chaincancelledfinalizedmarked_invalidprefixed_hash
02021-03-15T01:24:43.2596522022-02-22T22:22:00False164556852016157713790x53a174cf868ce142f00129a107b55c78261892f136ab...{'asset': {'id': '40913', 'address': '0x2a46f2...0x7be8076f4ea4a4ad08075c2508e481d6c946d12b{'user': {'username': 'Topnames'}, 'profile_im...{'user': {'username': 'NullAddress'}, 'profile......15716188216013657327630181200926559569323446352...280x6445ec0a1b190cbee568559fda23df5489df1e9b0af7...0x638b2fcc9e3c7402d9e4b029ae54d14b888d1ef88313...FalseFalseFalseFalse0x7fc5311e5073511ca7f32d8063df381a32d7555ab2d7...
12021-03-12T19:00:38.9587802021-03-19T17:59:07False161617674716155755350x4f66fbc8567a41b4cdf9bd7ffd831a2da5afb0e12b49...{'asset': {'id': '40913', 'address': '0x2a46f2...0x7be8076f4ea4a4ad08075c2508e481d6c946d12b{'user': {'username': 'Topnames'}, 'profile_im...{'user': {'username': 'NullAddress'}, 'profile......14913420786899617201762700420977053882698301780...270xf6677e63624b02cb35438f7e0d38fc7f1cd2f91d838f...0x1c8bb2ac4d1cd9bb8193207d25f573b9069401739bf4...FalseFalseFalseFalse0x96e982d5f2fbc5d3cea60027100d3bb9a545ac3261a8...
22021-03-12T05:48:13.7049732022-03-12T05:47:00False164706402016155279810x48a0cc883f19e0433167823536dbd30470ddced05838...{'asset': {'id': '40913', 'address': '0x2a46f2...0x7be8076f4ea4a4ad08075c2508e481d6c946d12b{'user': {'username': 'GeorgeG'}, 'profile_img...{'user': {'username': 'NullAddress'}, 'profile......18852251840142406136855228920314264057258444192...270x2e88aff635fb7685ffbbc874d2c1b4a5b0c8dae1ec0e...0x1f93249665affa20005ff2383478ade5cb25ee23f103...FalseFalseFalseFalse0x2c788df099b41faa5a41c4faaccfc891cee68cfbb7d9...
32021-03-12T02:37:37.9490082021-03-19T02:37:14False161612143416155165470xe728c96b72415a6a1577cf7b31fb17489e4ee99d80f3...{'asset': {'id': '40913', 'address': '0x2a46f2...0x7be8076f4ea4a4ad08075c2508e481d6c946d12b{'user': {'username': 'grax'}, 'profile_img_ur...{'user': {'username': 'NullAddress'}, 'profile......12066717606738834710390009518241782982595864725...280x1b879f61bc37c89f3b1e040c2cf96cc7681c7b6987ae...0x5a14a3701278c6a467aab349c2e99cd13e1f7b73bff5...FalseFalseFalseFalse0xb75da0cb69732b0524bac30a12a38fa833f1118fe031...
42021-03-12T00:49:00.1677072021-03-18T23:48:49False161611132916155100360x45cd8f6a7ab1ec8c58471f9a6ff53a58fddb779affa2...{'asset': {'id': '40913', 'address': '0x2a46f2...0x7be8076f4ea4a4ad08075c2508e481d6c946d12b{'user': {'username': 'Pelvis'}, 'profile_img_...{'user': {'username': 'NullAddress'}, 'profile......17358784693878811987544611408906685300841401434...270xc38982bbc6ac622c5e93d79b34e7f99e7400dcc8c178...0x591d2b876544d8412b4de74d804a41409225cdfaee94...FalseFalseFalseFalse0xb8aca8417175fbdfa957ab65b35ce05fa5f41282e015...

5 rows × 42 columns

Making a copy of the larger dataframe with a subset of columns

In [11]:
df = df_orders[['created_date', 'order_hash', 'current_price', 'maker']].copy()
df.head()
Out[11]:
created_dateorder_hashcurrent_pricemaker
02021-03-15T01:24:43.2596520x53a174cf868ce142f00129a107b55c78261892f136ab...22000000000000000000.00000000{'user': {'username': 'Topnames'}, 'profile_im...
12021-03-12T19:00:38.9587800x4f66fbc8567a41b4cdf9bd7ffd831a2da5afb0e12b49...10420690000000000000.00000000{'user': {'username': 'Topnames'}, 'profile_im...
22021-03-12T05:48:13.7049730x48a0cc883f19e0433167823536dbd30470ddced05838...10000000000000.00000000000000{'user': {'username': 'GeorgeG'}, 'profile_img...
32021-03-12T02:37:37.9490080xe728c96b72415a6a1577cf7b31fb17489e4ee99d80f3...1000000000000000.000000000000{'user': {'username': 'grax'}, 'profile_img_ur...
42021-03-12T00:49:00.1677070x45cd8f6a7ab1ec8c58471f9a6ff53a58fddb779affa2...8880000000000000000.000000000{'user': {'username': 'Pelvis'}, 'profile_img_...

Then I'm going to parse out the 'username' from the maker column. The maker column is currently a dict so all I need to do is use a lambda function and swim down the dict keys and pull out the name value.

In [12]:
df['maker'] = df['maker'].apply(lambda x: x['user']['username'])
df.head()
Out[12]:
created_dateorder_hashcurrent_pricemaker
02021-03-15T01:24:43.2596520x53a174cf868ce142f00129a107b55c78261892f136ab...22000000000000000000.00000000Topnames
12021-03-12T19:00:38.9587800x4f66fbc8567a41b4cdf9bd7ffd831a2da5afb0e12b49...10420690000000000000.00000000Topnames
22021-03-12T05:48:13.7049730x48a0cc883f19e0433167823536dbd30470ddced05838...10000000000000.00000000000000GeorgeG
32021-03-12T02:37:37.9490080xe728c96b72415a6a1577cf7b31fb17489e4ee99d80f3...1000000000000000.000000000000grax
42021-03-12T00:49:00.1677070x45cd8f6a7ab1ec8c58471f9a6ff53a58fddb779affa2...8880000000000000000.000000000Pelvis

Then, believe it or not, there is a very precise number representing the number of ETH that this offer was made for. Below I'm doing some Pandas judo to 1) convert the price into a float (it's currently a string), divide by a bunch, and then round off the result to make it digestible.

In [13]:
df['current_price'] = (df['current_price'].astype(float) / 1000000000000000000).round(5)
df.head()
Out[13]:
created_dateorder_hashcurrent_pricemaker
02021-03-15T01:24:43.2596520x53a174cf868ce142f00129a107b55c78261892f136ab...22.00000Topnames
12021-03-12T19:00:38.9587800x4f66fbc8567a41b4cdf9bd7ffd831a2da5afb0e12b49...10.42069Topnames
22021-03-12T05:48:13.7049730x48a0cc883f19e0433167823536dbd30470ddced05838...0.00001GeorgeG
32021-03-12T02:37:37.9490080xe728c96b72415a6a1577cf7b31fb17489e4ee99d80f3...0.00100grax
42021-03-12T00:49:00.1677070x45cd8f6a7ab1ec8c58471f9a6ff53a58fddb779affa2...8.88000Pelvis

Finally, let's sort our values and few the whole thing. Now we can see the orders for Beeple's 5000 days on OpenSea.io

In [14]:
df.sort_values('created_date', ascending=False)
Out[14]:
created_dateorder_hashcurrent_pricemaker
02021-03-15T01:24:43.2596520x53a174cf868ce142f00129a107b55c78261892f136ab...22.00000Topnames
12021-03-12T19:00:38.9587800x4f66fbc8567a41b4cdf9bd7ffd831a2da5afb0e12b49...10.42069Topnames
22021-03-12T05:48:13.7049730x48a0cc883f19e0433167823536dbd30470ddced05838...0.00001GeorgeG
32021-03-12T02:37:37.9490080xe728c96b72415a6a1577cf7b31fb17489e4ee99d80f3...0.00100grax
42021-03-12T00:49:00.1677070x45cd8f6a7ab1ec8c58471f9a6ff53a58fddb779affa2...8.88000Pelvis
52021-03-11T17:09:05.5344070xaf0ed98865f214f33f19256c03789211656cf72ffaf9...0.06935l3en
62021-03-11T10:59:08.8601500x77c770e4c5463c0d25931ff87aa0d4f28841c63dc6a4...0.02400Jawz
72021-03-10T10:44:01.9507200xd337df66fcf3a0eaed18468818b589f2721d392bcd18...0.00000Kevingusa
82021-03-10T10:38:09.1228030x0c7b767fe1bc99f0bde9e3d894107223abf85f26db7d...0.00000Kevingusa
92021-03-10T07:43:22.7374420xccd62ae4e9eee129be3f7bc7def3326818d1d57bad55...0.02300AaronLeupp
102021-03-10T01:27:59.8027100xece9d7f3da1561820de911aada3e4e44600497a66214...0.02000None
112021-03-09T22:37:48.2525810x3d72d54550c8d1463777eb0c12f48a2410bc36e87bae...0.00054Crypto_Riv
In [ ]: