Definitely, using BigQuery for what BigQuery is really good at (big scans and cost-based joins) is nearly always a good idea. A strong endorsement of Ankur's answer.
Pushing the right amount of work into a database is an art, however -- there are some scenarios where you'd rather scan in BQ and join in Beam because the join result is very large and you can better filter it in Beam, or because you need to do some pre-join-filtering based on an external API call (and you don't want to load the results of that API call into BigQuery)... I've only seen a few, rare, cases of the latter. Thanks, Dan On Sun, Feb 5, 2017 at 9:19 PM, Prabeesh K. <[email protected]> wrote: > Hi Ankur, > > Thank you for your response. > > On 5 February 2017 at 23:59, Ankur Chauhan <[email protected]> wrote: > >> I have found doing joins in bigquery using sql is a lot faster and easier >> to iterate upon. >> >> >> Ankur Chauhan >> On Sat, Feb 4, 2017 at 22:05 Prabeesh K. <[email protected]> wrote: >> >>> Hi, >>> >>> Which is the better way to join two tables in apache beam? >>> >>> Regards, >>> Prabeesh K. >>> >> >
