Re: [HACKERS] [PATCH] Call RelationDropStorage() for broader range of object drops.

2017-09-13 Thread Hadi Moshayedi
To provide more context, in cstore_fdw creating the storage is easy, we only need to hook into CREATE FOREIGN TABLE using event triggers. Removing the storage is not that easy, for DROP FOREIGN TABLE we can use event triggers. But when we do DROP EXTENSION, the event triggers don't get fired (becau

Re: [HACKERS] [PATCH] Call RelationDropStorage() for broader range of object drops.

2017-09-13 Thread Hadi Moshayedi
On Wed, Sep 13, 2017 at 12:12 AM, Michael Paquier wrote: > > Foreign tables do not have physical storage assigned to by default. At > least heap_create() tells so, create_storage being set to false for a > foreign table. So there is nothing to clean up normally. Or is > cstore_fdw using directly h

[HACKERS] [PATCH] Call RelationDropStorage() for broader range of object drops.

2017-09-12 Thread Hadi Moshayedi
Motivation for this patch is that some FDWs (notably, cstore_fdw) try utilizing PostgreSQL internal storage. PostgreSQL assigns relfilenode's to foreign tables, but doesn't clean up storage for foreign tables when dropping tables. Therefore, in cstore_fdw we have to do some tricks to handle droppin

[HACKERS] [PATCH] Make ExplainOpenGroup()/ExplainCloseGroup() available for extensions.

2017-07-25 Thread Hadi Moshayedi
Hello Hackers, The attached patch moves declarations of ExplainOpenGroup()/ExplainCloseGroup() from explain.c to explain.h. This can be useful for extensions that need explain groups in their custom-scan explain output. For example, Citus uses groups in its custom explain outputs [1]. But it ach

Re: [HACKERS] [PATCH] Make ExplainBeginGroup()/ExplainEndGroup() public.

2017-07-21 Thread Hadi Moshayedi
Title should have been "Make ExplainOpenGroup()/ExplainCloseGroup() public.". Sorry for the misspell.

[HACKERS] [PATCH] Make ExplainBeginGroup()/ExplainEndGroup() public.

2017-07-21 Thread Hadi Moshayedi
Hello, The attached patch moves declarations of ExplainBeginGroup()/ExplainEndGroup() from explain.c to explain.h. This can be useful for extensions that need explain groups in their custom-scan explain output. For example, Citus uses groups in its custom explain outputs [1]. But it achieves it

Re: [HACKERS] Faster array_length()

2014-05-02 Thread Hadi Moshayedi
Thanks for looking into this. > With that in mind, I was surprised that your test case showed any > improvement at all --- it looks like the arrays aren't getting compressed > for some reason. > > You are right, it seems that they were not getting compressed, probably because the arrays were "seq

[HACKERS] Faster array_length()

2014-05-02 Thread Hadi Moshayedi
Hello, The attached patch improves the performance of array_length() by detoasting only the overhead part of the datum. Here is a test case: postgres=# create table array_length_test as select array_agg(a) a from generate_series(1, 1) a, generate_series(1, 1) b group by b; Without the p

Re: [HACKERS] Proposal for Merge Join for Non '=' Operators

2014-04-29 Thread Hadi Moshayedi
Hello Dilip, Query: select count(*) from t1,t2 where t1.b 12000; > > Test Result: > Nest Loop Join with Index Scan : 1653.506 ms > Sort Merge Join for (seq scan) : 610.257ms > > This looks like a great improvement. Repeating Nicolas's question, d

Re: [HACKERS] A question about code in DefineRelation()

2014-04-25 Thread Hadi Moshayedi
> > On second thought I noticed that that makes CREATE FOREIGN TABLE include > an OID column in newly-created foreign tables wrongly, when the > default_with_oids parameter is set to on. Please find attached a patch. > > The fix makes sense to me, since in ALTER TABLE SET WITH OIDS we check that t

Re: [HACKERS] PostgreSQL Columnar Store for Analytic Workloads

2014-04-08 Thread Hadi Moshayedi
Hi Stefan, On Tue, Apr 8, 2014 at 9:28 AM, Stefan Keller wrote: > Hi Hadi > > Do you think that cstore_fd*w* is also welll suited for storing and > retrieving linked data (RDF)? > I am not very familiar with RDF. Note that cstore_fdw doesn't change the query language of PostgreSQL, so if your

[HACKERS] PostgreSQL Columnar Store for Analytic Workloads

2014-04-03 Thread Hadi Moshayedi
Dear Hackers, We at Citus Data have been developing a columnar store extension for PostgreSQL. Today we are excited to open source it under the Apache v2.0 license. This columnar store extension uses the Optimized Row Columnar (ORC) format for its data layout, which improves upon the RCFile forma

[HACKERS] PostgreSQL Columnar Store for Analytic Workloads

2014-04-03 Thread Hadi Moshayedi
Dear Hackers, We at Citus Data have been developing a columnar store extension for PostgreSQL. Today we are excited to open source it under the Apache v2.0 license. This columnar store extension uses the Optimized Row Columnar (ORC) format for its data layout, which improves upon the RCFile forma

[HACKERS] pglz_decompress()

2014-02-27 Thread Hadi Moshayedi
Hello, The comments in pg_lzcompress.c say that: * If VARSIZE(x) == rawsize + sizeof(PGLZ_Header), then the data * is stored uncompressed as plain bytes. Thus, the decompressor * simply copies rawsize bytes from the location after the * header to the destination. But pg_lzdecompress doesn't

[HACKERS] Cost estimation in foreign data wrappers

2014-02-21 Thread Hadi Moshayedi
Hello, There is a callback function in fdw's which should also set estimates for startup and total costs for each path. Assume a fdw adds only one path (e.g. in file_fdw). I am trying to understand what can go wrong if we do a bad job in estimating these costs. Since we have only one scan path he

Re: [HACKERS] Improving avg performance for numeric

2013-08-28 Thread Hadi Moshayedi
Hello, > int, float, double 26829 ms (26675 ms) -- 0.5% slower .. statistic error .. > cleaner code > numeric sum 6490 ms (7224 ms) -- 10% faster > numeric avg 6487 ms (12023 ms) -- 46% faster I also got very similar results. On the other hand, initially I was receiving sigsegv's whenever I wan

Re: [HACKERS] Improving avg performance for numeric

2013-03-20 Thread Hadi Moshayedi
Hi Tom, Tom Lane wrote: > After thinking about that for awhile: if we pursue this type of > optimization, what would probably be appropriate is to add an aggregate > property (stored in pg_aggregate) that allows direct specification of > the size that the planner should assume for the aggregate's

Re: [HACKERS] Improving avg performance for numeric

2013-03-19 Thread Hadi Moshayedi
I am not sure how this works, but I also changed numeric sum(), and the views in question had a numeric sum() column. Can that have any impact? I am going to dig deeper to see why this happens. On Tue, Mar 19, 2013 at 6:25 PM, Tom Lane wrote: > Kevin Grittner writes: > > Hadi Moshaye

Re: [HACKERS] Improving avg performance for numeric

2013-03-18 Thread Hadi Moshayedi
se a only preallocated > buffers. A speed of numeric is more important now, because there are > more and more warehouses, where CPU is botleneck. > > Regards > > Pavel > > > 2013/3/18 Hadi Moshayedi : > > Hi Pavel, > > > > Thanks a lot for your feedback.

Re: [HACKERS] Improving avg performance for numeric

2013-03-18 Thread Hadi Moshayedi
e: > 2013/3/16 Hadi Moshayedi : > > Revisiting: > > http://www.postgresql.org/message-id/45661be7.4050...@paradise.net.nz > > > > I think the reasons which the numeric average was slow were: > > (1) Using Numeric for count, which is slower than int8 to incre

[HACKERS] Improving avg performance for numeric

2013-03-15 Thread Hadi Moshayedi
Revisiting: http://www.postgresql.org/message-id/45661be7.4050...@paradise.net.nz I think the reasons which the numeric average was slow were: (1) Using Numeric for count, which is slower than int8 to increment, (2) Constructing/deconstructing arrays at each transition step. This is also disc