Database Administrators Stack Exchange
Database Administrators Stack Exchange

Questions tagged [google-bigquery]

Google BigQuery is a web service that lets you do interactive analysis of massive datasets—analyzing billions of rows in seconds. Scalable and easy to use, BigQuery lets developers and businesses tap into powerful data analytics on demand.

Google BigQuery is a web service that lets you do interactive analysis of massive datasets—up to billions of rows. Scalable and easy to use, BigQuery lets developers and businesses tap into powerful data analytics on demand.

Official sites:

Other sites for related topics and discussion:

60 questions
4
votes
2 answers

How to query for certain text in log messages from Google App Engine that are exported to BigQuery?

I have used the 'Create Export' in the Logging section of Google Cloud Platform to export log entries into Big Query for analysis Here is part of the structure of the table in BigQuery that logs the log entry I want to search for entries that…
Anthony Kong
  • 141
  • 4
4
votes
2 answers

How we can do full text search and facets stuffs on BigQuery?

In the future, we will have millions ofs record in our web product. So we had used BigQuery for data storage and analysis. We have to build filters on our search page like Flipkart and Amazon provide product filters. Basically, we need different…
Lalit Kumar Maurya
  • 143
  • 1
  • 6
3
votes
0 answers

Alternatives for storing 365TB of info

I need to upload daily CSV dumps to a blackbox of my choice, and it should be possible to run queries or create filters over the uploaded data. Daily dumps weight around 1TB and are basically id, datetime, latitude and longitude columns. It's…
ffflabs
  • 345
  • 1
  • 3
  • 11
3
votes
1 answer

Equivalent idioms in Oracle?

I was reading this article that is about Google BigQuery: Exploring a powerful SQL pattern: ARRAY_AGG, STRUCT and UNNEST They use a couple of functions that I am trying to figure out what the Oracle equivalents are, if they even exist. I am…
user68575
3
votes
1 answer

Reference outer variables in nested select in bigquery vs mysql

I run following request in MySQL and it works, but results in an error with bigquery standard SQL. $sql = <<
Alexander Smirnov
  • 133
  • 1
  • 5
3
votes
1 answer

Replacing an aggregation system of statistics

We currently have a database under MySQL, storing aggregated statistics in different tables (recent hours, hours, days, months). The tables are updated by workers running at different rates depending on the freshness required for the data. Then…
Dysosmus
  • 131
  • 1
2
votes
1 answer

Can you use BigQuery to run on top of Bigtable

I need to run BigQuery on top of Bigtable live, not as an export. I have found the information stating it was in beta but only as an export function. I would like to run BigQuery against Bigtable data without exporting. Has anyone done this or is it…
cmydata
  • 21
  • 3
1
vote
2 answers

Counting unique records across 2 columns and computing a score based on boolean values

I have a table that looks like this in Google Big Query: custID InteractionDate Purchased? Sales 1 20150312 F 0 1 20150312 T 200 1 20150314 T 150 2 20150221 T …
xupv5
  • 133
  • 1
  • 4
1
vote
1 answer

Aggregating two queries and returning a COUNT of some permutations

Hopefully the title can make sense to somebody, so I can get somebody working with a database (BigQuery in my case) to look into into. Given two quarters, I need to find out the # of orders placed by each consumer for each of the two. I need also to…
Andrea Moro
  • 255
  • 2
  • 4
  • 14
1
vote
1 answer

Aggregate SCD Type 2 data "as of" each day

Problem When working with SCD type 2 data it's easy to see the state of a table "as of" a given point in time by using your date columns, eg: valid_from and valid_to. For example: select * from table where '2023-11-01' between valid_from and…
camtech
  • 143
  • 6
1
vote
1 answer

Using COUNT returns the same value when counting two different columns

I am new to SQL and am using it for a school project and would like some help. I have a table with multiple columns about a fictional company that provides a bike sharing service. One of the rows I am trying to count is called member_casual which…
YellingYeti
  • 11
  • 2
1
vote
0 answers

Convert any string to url valid percent encoding in BigQuery

I am trying to convert any string with any set of special characters into a valid url of the format below. In Bigquery Example: /artwork-v2/-̴̕ι-̶͔͛n̴e̷p̸u̴̒n̵uś̵̥o̵̙̾rt̷͗um̶̹͐-20380 encodes…
Keegan Ead
  • 11
  • 1
1
vote
0 answers

What is the appropriate SQL statement/s to produce a stacked bar chart for multiple random variable distributions?

The Objective Graph the difference between the probability distributions of sets of random variables that come from a number of different sources using a stacked bar chart (or another type of graph suited for such visualization). The Problem I'm not…
pilcrowpipe
  • 111
  • 2
1
vote
1 answer

how to get weekly unbound retention from bigquery?

I Need to get the weekly retention users. If user has made the transaction in week 4, that user is there in Week 0, week 1, week 2, week 3, week 4. If User made transaction in week 0, and week 3 then user will be there in Week 0, week 1, week 2,…
Mallikarjun
  • 13
  • 4
1
vote
0 answers

Keeping the same order of selected records while inserting to another table

I am running a below query in big query insert into `test-project.temp_test_dataset.cluster_records` (pm,col1,col2,col3,total) select pm,col1,col2,col3,total from ( select 'ingest' as pm,col1,col2,col3,col4,col5,sum(col3+col4) as total from…
vikrant rana
  • 291
  • 1
  • 8
  • 18
1
2 3 4