10:09 GMT+09:00 Kouhei Kaigai kai...@ak.jp.nec.com:
On 8 May 2014 22:55, Tom Lane t...@sss.pgh.pa.us wrote:
We're past the prototyping stage and into productionising what
we know works, AFAIK. If that point is not clear, then we need
to discuss that first.
OK, I'll bite: what
On 8 May 2014 22:55, Tom Lane t...@sss.pgh.pa.us wrote:
We're past the prototyping stage and into productionising what we
know works, AFAIK. If that point is not clear, then we need to
discuss that first.
OK, I'll bite: what here do we know works? Not a damn thing AFAICS;
it's all
On Wed, May 7, 2014 at 4:01 AM, Simon Riggs si...@2ndquadrant.com wrote:
Agreed. My proposal is that if the planner allows the lookaside to an
FDW then we pass the query for full execution on the FDW. That means
that the scan, aggregate and join could take place via the FDW. i.e.
Custom
I also think that there are really two separate problems here: getting
the executor to call a custom scan node when it shows up in the plan
tree; and figuring out how to get it into the plan tree in the first
place. I'm not sure we've properly separated those problems, and I'm
not sure
On Thu, May 8, 2014 at 4:43 PM, Tom Lane t...@sss.pgh.pa.us wrote:
I thought that the executor side of his patch wasn't in bad shape.
The real problems were in the planner, and indeed largely in the backend
part of the planner where there's a lot of hard-wired logic for fixing
up
On Thu, May 8, 2014 at 6:34 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote:
Umm... I'm now missing the direction towards my goal.
What approach is the best way to glue PostgreSQL and PGStrom?
I haven't really paid any attention to PGStrom. Perhaps it's just that I
missed it, but I would
So it seems reasonable to have a way to define/declare what is
possible and what is not. But my take is that adding a new column to
pg_operator for every CustomJoin node is probably out of the question,
hence my suggestion to list the operators we know it can work with.
It does seem
* Kouhei Kaigai (kai...@ak.jp.nec.com) wrote:
I initially intended to allow extensions to add their custom-path
based on their arbitrary decision, because the core backend cannot
have expectation towards the behavior of custom-plan.
However, of course, the custom-path that replaces built
* Kouhei Kaigai (kai...@ak.jp.nec.com) wrote:
I didn't ask this before but it's been on my mind for a while- how
will this work for custom data types, ala the 'geometry' type from
PostGIS?
There's user-provided code that we have to execute to check equality
for those, but they're
On 7 May 2014 02:05, Kouhei Kaigai kai...@ak.jp.nec.com wrote:
Prior to the development cycle towards v9.5, I'd like to reopen the
discussion of custom-plan interface. Even though we had lots of
discussion during the last three commit-fests, several issues are
still under discussion. So
] Custom Plan API
On 7 May 2014 08:17, Kouhei Kaigai kai...@ak.jp.nec.com wrote:
Let me clarify. This mechanism allows to add alternative scan/join
paths including built-in ones, not only custom enhanced plan/exec node,
isn't it?
Probably, it is a variation of above proposition if we
Let me list up the things to be clarified / developed randomly.
* Join replacement by FDW; We still don't have consensus about join
replacement
by FDW. Probably, it will be designed to remote-join implementation
primarily,
however, things to do is similar. We may need to revisit
* ForeignScan node that is not associated with a particular
foreign-table.
Once we try to apply ForeignScan node instead of Sort or
Aggregate,
existing
FDW implementation needs to be improved. These nodes scan on a
materialized
relation (generated on the fly), however,
Andres Freund and...@2ndquadrant.com writes:
What I think this discussion shows that this patch isn't ready for
9.4. The first iteration of the patch came in 2013-11-06. Imo that's
pretty damn late for a relatively complex patch. And obviously we
don't have agreement on the course
On Tue, Apr 15, 2014 at 10:44 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Mon, Apr 14, 2014 at 4:43 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Yeah. After a fast review of the custom-scan and cache-scan
patches, it seems to me that my original fears
On 24 March 2014 10:25, Kouhei Kaigai kai...@ak.jp.nec.com wrote:
Brief summary of the current approach that has been revised from my
original submission through the discussion on pgsql-hackers:
The plannode was renamed to CustomPlan, instead of CustomScan, because
it dropped all
Simon Riggs si...@2ndquadrant.com writes:
[ assorted comments about custom-scan patch, but particularly ]
* The prune hook makes me feel very uneasy. It seems weirdly specific
implementation detail, made stranger by the otherwise lack of data
maintenance API calls. Calling that for
version.
Thanks,
--
NEC OSS Promotion Center / PG-Strom Project
KaiGai Kohei kai...@ak.jp.nec.com
-Original Message-
From: pgsql-hackers-ow...@postgresql.org
[mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Kouhei Kaigai
Sent: Monday, March 17, 2014 9:29 AM
To: Kaigai Kouhei(海外
for cache-only
table scan?)
On Mon, Mar 17, 2014 at 11:45 AM, Kouhei Kaigai kai...@ak.jp.nec.com
wrote:
Hello,
The attached patches are revised ones according to the latest
custom-plan interface patch (v11).
The cache-scan module was re-implemented on the newer interface, and
also I
I hacked on this for awhile, but there remain two matters on which I'm
uncertain about the right way forward.
(1) To acquire the old tuple for UPDATE/DELETE operations, the patch closely
parallels our handling for INSTEAD OF triggers on views. It adds a wholerow
resjunk attribute, from
Thanks for your efforts!
Head patched
Diff
Select - 500K772ms2659ms-200%
Insert - 400K 3429ms 1948ms 43% (I am
not sure how it improved in this case)
delete - 200K
KaiGai Kohei kai...@ak.jp.nec.com
-Original Message-
From: pgsql-hackers-ow...@postgresql.org
[mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Kouhei Kaigai
Sent: Wednesday, March 12, 2014 3:26 PM
To: Haribabu Kommi; Kohei KaiGai
Cc: Tom Lane; PgHacker; Robert Haas
Subject: Re
I was thinking more like
typedef struct CustomPathFuncs {
const char *name; /* used for debugging purposes only */
NodeCopy_function node_copy;
NodeTextOut_function node_textout;
... etc etc etc ...
} CustomPathFuncs;
typedef struct CustomPath {
Tom, thanks for your detailed comments.
I apologize for not having paid much attention to this thread so far.
It kept getting stuck on my to look at later queue. Anyway, I've taken
a preliminary look at the v7 patch now.
While the patch seems roughly along the lines of what we talked about
Kouhei Kaigai kai...@ak.jp.nec.com writes:
* Please drop the whole register_custom_provider/get_custom_provider
API.
One thing I was worrying about is how copyObject() and nodeToString()
support set of function pointer tables around custom-scan node,
however, you suggested to change
As I mentioned
up-thread, I'd really like to see FDW join push-down, FDW aggregate
push-down, parallel query execution, and parallel remote-FDW execution
and I don't see this CustomScan approach as the right answer to any of
those.
In accordance with the above, what I'd like to see
* Kouhei Kaigai (kai...@ak.jp.nec.com) wrote:
This regular one means usual tables. Even though custom implementation
may reference self-managed in-memory cache instead of raw heap, the
table pointed in user's query shall be a usual table.
In the past, Hanada-san had proposed
If you're looking to just use GPU acceleration for improving
individual queries, I would think that Robert's work around backend
workers would be a more appropriate way to go, with the ability to
move a working set of data from shared buffers and on-disk
representation of a relation
Thanks for the information, I will apply other patches also and
start testing.
When try to run the pgbench test, by default the cache-scan plan is not
chosen because of more cost. So I increased the cpu_index_tuple_cost to
a maximum value or by turning off index_scan, so that the
* Kouhei Kaigai (kai...@ak.jp.nec.com) wrote:
IIUC, his approach was integration of join-pushdown within FDW APIs,
however, it does not mean the idea of remote-join is rejected.
For my part, trying to consider doing remote joins *without* going through
FDWs is just nonsensical. What
* Kouhei Kaigai (kai...@ak.jp.nec.com) wrote:
I (plan to) use custom-scan of course. Once a relation is referenced
and optimizer decided GPU acceleration is cheaper, associated custom-
scan node read the data from underlying relation (or in-memory cache
if exists) then move to the shared
think it is a reasonable solution, however, I'm not 100% certain
whether people have more graceful idea to implement it.
If you have comments around above two topic, or others, please give
your ideas.
Thanks,
2014-01-28 9:14 GMT+09:00 Kouhei Kaigai kai
* Kouhei Kaigai (kai...@ak.jp.nec.com) wrote:
Instead of custom node, it might be better idea to improve FDW
infrastructure to push join. For the starters, is it possible for
the custom scan node hooks to create a ForeignScan node? In general,
I think, it might be better
* Kouhei Kaigai (kai...@ak.jp.nec.com) wrote:
Yes, the part-1 patch provides a set of interface portion to interact
between the backend code and extension code. Rest of part-2 and part-3
portions are contrib modules that implements its feature on top of
custom-scan API.
Just to come
/* Hook for plugins to add custom join path, in addition to default
ones */ typedef void (*add_join_path_hook_type)(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
8. I am not able to find a protection mechanism in insert/delete
and etc of
a tuple in Ttree. As this is a shared memory it can cause problems.
For design simplification, I put a giant lock per columnar-cache.
So, routines in cscan.c acquires exclusive
Hello,
I initially tried to keep track of them by allocating read pointers on the
tuple store, but it turned out to be so expensive that I had to find another
way (24bytes per stored tuple, which are not reclaimable until the end of
the transaction).
What do you think about this approach ?
Kouhei Kaigai kai...@ak.jp.nec.com writes:
Let me ask an elemental question. What is the reason why inheritance
expansion logic should be located on the planner stage, not on the
tail of rewriter?
I think it's mostly historical. You would however have to think of a way
to preserve
Hackers,
Is somebody available to volunteer to review the custom-scan patch?
Even though Hanada-san acknowledged before, it seems to me this patch
has potentially arguable implementations. Even if you have enough time
to review whole of the code, it helps me if you can comment on the
following
Hi Stephen,
Thanks for your comments.
* Kouhei Kaigai (kai...@ak.jp.nec.com) wrote:
Is somebody available to volunteer to review the custom-scan patch?
I looked through it a bit and my first take away from it was that the patches
to actually use the new hooks were also making more changes
AFAICS the only area of objection is the handling of inherited
relations, which occurs within the planner in the current patch. I can
see that would be a cause for concern since the planner is pluggable
and it would then be possible to bypass security checks. Obviously
installing a new
I wonder what shall be the cases when foreign table is on a server which
does not support *all* SQL features.
Does a FDW need to have the possible inherit options mentioned in its
documentation for this patch?
The answer is no, in my understanding. The altering operation simply
Hello,
I checked the latest updatable security barrier view patch.
Even though I couldn't find a major design problem in this revision,
here are two minor comments below.
I think, it needs to be reviewed by committer to stick direction
to implement this feature. Of course, even I know Tom argued
401 - 443 of 443 matches
Mail list logo
