How To Fix: BigQuery Data Set Too Big to Download

/

blog-big-big-query

There are many different quirks to BigQuery that can be frustrating if you don’t know the solution. Check out this post and this other post by some awesome coworkers to know more about getting started with BigQuery. For an example of one of those quirks, click here.

Downloading Big Big Query Data Sets

One problem that comes up often, is not being able to download a returned data set from the BigQuery Web UI because it is too large. This blog post will talk about how to solve that issue. It’s not too complicated, it’s just a matter of knowing how to do it!

Create A Place To Store It

First, make a Google Cloud Storage bucket

Go here and click “Create Bucket.”

Create Bucket

Create the bucket. This name is global across all users of Google Cloud Storage so make sure it is unique and don’t forget this name!

UniqueName2

UniqueName2

Now go back over to BigQuery.

Export the Table

When the output of the query is saved as a table, it is then possible to use the little arrow to see a second menu. To make this arrow visible, mouse over the table and then click ‘Export table.’

big-query-export-table

Where Should We Save It?

Next, enter bucket name they created earlier/file name to export to/.csv. Following the examples above, the cloud storage URI would look like this:

Bucket

Watch It Work

Now it should show that the table is being “extracted” and 1 job is running.

big-query-extract

Look For It

Once it’s finished, go back to the Google Cloud Storage from earlier and the export is shown in the GCS browser.

bucketsaved

Download It!

Now it is possible to click on the file and download it. And VoliĆ ! Easy as that!

It should be noted that this file will cost money to store. Google Cloud Storage is not included in the $500 credit that comes with Google Analytics 360. However, it’s $0.026 per GB per month so that’s not too bad! Check out the pricing.

Kathleen is a Junior Consultant at LunaMetrics. Before coming to LunaMetrics, Kathleen spent several years living between Florence, Paris, New York City, and Pittsburgh (her all time favorite). Ready for a new adventure, she traded in her romance languages for coding languages in order to become an apprentice. She is learning all there is to know about data analytics and is luckily surrounded by many smart people willing to teach her!

Contact Us.

LunaMetrics

24 S. 18th Street, Suite 100,
Pittsburgh, PA 15203

Follow Us

1.877.220.LUNA

1.412.381.5500

getinfo@lunametrics.com

Questions?
We'll get back to you
in ONE business day.