I use Postgres. Replicate a subset of your entity data to a datastore that
supports aggregations. Assuming your dataset fits in a traditional RDBMS,
they’re awesome for aggregations and ad-hoc queries.
The datastore makes an awesome primary datastore because it is zero
maintenance and never breaks, but it isn’t the only tool in the toolbox.
Use dedicated “slave” indexes for special purposes like fulltext search and
aggregation. I use the task queue for replication; whenever an entity
changes in a material way, my code transactionally enqueues a sync-task
that syncs the relevant state of that entity to the appropriate index.
You might be asking yourself why not just use a RDBMS in the first place?
It’s a reasonable question, and for many apps it is the right one. But
running an RDBMS as a primary store has a cost - you have to be really
careful with it. Running a DDL statement that locks up the db is a minor
inconvenience when it takes down your reports pages for a half hour; if it
takes down your sales flow it’s catastrophic. Personally I like the
two-store solution, but I had to build some infrastructure to make the
syncing process easier.
If your dataset doesn’t fit in an RDBMS or has load requirements that make
relational databases untenable, you can still use BigQuery and similar. You
can delete/change records by adding new records that “cancel out” the old
records. It’s not as pretty as an update but adds up just the same.
Jeff
On Thu, Jan 26, 2017 at 4:17 AM, Rajesh Gupta <
rajesh.gu...@veersoftsolutions.com> wrote:
> Hello,
>
> No, we cannot do bigquery replication, as invoices may get deleted and
> updated.
>
> We do these reports in a task, and tasks write to memcache the report data.
>
> Any json technologies here?. Can the invoice list be written using
> jackson as json, and use any json query technologies.
>
> On Thu, Jan 26, 2017 at 2:51 PM, Nicholas Okunew <naoku...@gmail.com>
> wrote:
>
>> 1. You need to remove all doubles and use bigdecimal or joda money. If
>> you don't know why, google it.
>>
>> 2. If you want to do adhoc reporting, consider replicating to bigquery
>> and running from there. If you don't want to do that, queries in batches on
>> backends is the way to go, but this kind of reporting is the datastores
>> weakspot.
>>
>> On Thursday, 26 January 2017, Rajesh Gupta <rajesh.gupta@veersoftsolution
>> s.com> wrote:
>>
>>> Hi,
>>>
>>> I have the following Entities (Shown in objectify style)
>>> I want to do several reports for this data with different invoice date
>>> ranges.
>>> - Sales by customer
>>> - Product Qty sold by product
>>> - Product value by product
>>>
>>> The reports can be filtered with any Date range like 'Last Month', 'Last
>>> Qtr', 'Year to date' or any custom date range.
>>>
>>> The no of invoices fetched can be from 100-10000, depending on the date
>>> range.
>>>
>>> What is the good way to do such aggregate reports.?
>>>
>>> So far, we have used looping with hashmaps. But this style of reports
>>> will increase and the data will also increase. I am looking for more
>>> efficient data structures or schemas.
>>>
>>> @Entity
>>> class SalesInvoice {
>>> @Id Long id;
>>> Key<Customer> customerKey;
>>>
>>> List<LineItem> lineItems
>>>
>>> double total;
>>> Key<SalesPerson> salesPersonKey;
>>> Date invoiceDate;
>>> }
>>>
>>> @Embed
>>> class LineItem {
>>> Key<Product> productKey;
>>> double qty;
>>> double price;
>>> double tax;
>>> double totalPrice
>>> }
>>>
>>> --
>>> Regards,
>>> Rajesh
>>> *www.VeersoftSolutions.com <http://www.VeersoftSolutions.com>*
>>> *www.GainERP.com <https://www.gainerp.com>*
>>> *Accounting/Inventory/Orders/Sales/Purchase on Google Cloud Platform and
>>> Mobile*
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "objectify-appengine" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to objectify-appengine+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "objectify-appengine" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to objectify-appengine+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Regards,
> Rajesh
> *www.VeersoftSolutions.com <http://www.VeersoftSolutions.com>*
> *www.GainERP.com <https://www.gainerp.com>*
> *Accounting/Inventory/Orders/Sales/Purchase on Google Cloud Platform and
> Mobile*
>
> --
> You received this message because you are subscribed to the Google Groups
> "objectify-appengine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to objectify-appengine+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit
https://groups.google.com/d/msgid/google-appengine/CADK-0uj7zgDfsumLndoi8EGKT1A2LtEJwWbaVcsNY46QB4rYmg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
