[HACKERS] Why my C function is called twice?

[Billow Gao]

A very simple C function which I copied from the manual.
And I found that it's called twice.

Even in the function:

if (SRF_IS_FIRSTCALL()) {
ereport(INFO, (errcode(ERRCODE_IO_ERROR), errmsg("1")));
}

An example output. You will find two INFO: 1 there..

Why a function is called twice and how to prevent this problem?

Thanks

Billow

test=# select * from retcomposite(1,48);
INFO:  1
INFO:  2
INFO:  3
INFO:  1
INFO:  4
INFO:  2
INFO:  2
INFO:  5
INFO:  3
INFO:  4
INFO:  2
INFO:  5
 f1 | f2 | f3
++-
 48 | 96 | 144
(1 row)



===
Code..
===
/*
CREATE OR REPLACE FUNCTION retcomposite(integer, integer)
RETURNS SETOF __retcomposite
AS 'test.so', 'retcomposite'
LANGUAGE C IMMUTABLE STRICT;
*/

// PostgreSQL includes
#include "postgres.h"
#include "fmgr.h"

// Tuple building functions and macros
#include "access/heapam.h"
#include "funcapi.h"
#include "utils/builtins.h"

#include 
#include 
#include 
#include 

#define _textout(str) DatumGetPointer(DirectFunctionCall1(textout,
PointerGetDatum(str)))

#ifdef PG_MODULE_MAGIC
PG_MODULE_MAGIC;
#endif


PG_FUNCTION_INFO_V1(retcomposite);

Datum
retcomposite(PG_FUNCTION_ARGS)
{
  FuncCallContext *funcctx;
  int call_cntr;
  int max_calls;
  TupleDesc tupdesc;
  AttInMetadata *attinmeta;

  /* stuff done only on the first call of the function */
  if (SRF_IS_FIRSTCALL()) {
MemoryContext oldcontext;

ereport(INFO, (errcode(ERRCODE_IO_ERROR), errmsg("1")));

/* create a function context for cross-call persistence
 */
funcctx = SRF_FIRSTCALL_INIT();

/* switch to memory context appropriate for multiple
   function calls */
oldcontext =
MemoryContextSwitchTo(funcctx->
  multi_call_memory_ctx);

/* total number of tuples to be returned */
funcctx->max_calls = PG_GETARG_UINT32(0);

/* Build a tuple descriptor for our result type */
if (get_call_result_type(fcinfo, NULL, &tupdesc) !=
TYPEFUNC_COMPOSITE)
  ereport(ERROR,
  (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
   errmsg
   ("function returning record called in context "
"that cannot accept type record")));

/* generate attribute metadata needed later to produce
   tuples from raw C strings */
attinmeta = TupleDescGetAttInMetadata(tupdesc);
funcctx->attinmeta = attinmeta;

MemoryContextSwitchTo(oldcontext);
  }

ereport(INFO, (errcode(ERRCODE_IO_ERROR), errmsg("2")));


  /* stuff done on every call of the function */
  funcctx = SRF_PERCALL_SETUP();

  call_cntr = funcctx->call_cntr;
  max_calls = funcctx->max_calls;
  attinmeta = funcctx->attinmeta;

  if (call_cntr < max_calls) {  /* do when there is more
   left to send */
char **values;
HeapTuple tuple;
Datum result;
ereport(INFO, (errcode(ERRCODE_IO_ERROR), errmsg("3")));

/* Prepare a values array for building the returned
   tuple. This should be an array of C strings which
   will be processed later by the type input functions. */
values = (char **) palloc(3 * sizeof(char *));
values[0] = (char *) palloc(16 * sizeof(char));
values[1] = (char *) palloc(16 * sizeof(char));
values[2] = (char *) palloc(16 * sizeof(char));

snprintf(values[0], 16, "%d", 1 * PG_GETARG_INT32(1));
snprintf(values[1], 16, "%d", 2 * PG_GETARG_INT32(1));
snprintf(values[2], 16, "%d", 3 * PG_GETARG_INT32(1));

/* build a tuple */
tuple = BuildTupleFromCStrings(attinmeta, values);

/* make the tuple into a datum */
result = HeapTupleGetDatum(tuple);

/* clean up (this is not really necessary) */
pfree(values[0]);
pfree(values[1]);
pfree(values[2]);
pfree(values);

ereport(INFO, (errcode(ERRCODE_IO_ERROR), errmsg("4")));


SRF_RETURN_NEXT(funcctx, result);
  } else {  /* do when there is no more left */
ereport(INFO, (errcode(ERRCODE_IO_ERROR), errmsg("5")));

SRF_RETURN_DONE(funcctx);
  }
}

[HACKERS] How to write a c-function to return multiple bytea rows

[Billow Gao]

I can return multiple strings w/o problem.

But if I tried to return multiple bytea rows. It only return 10 rows with
empty data.
Please see the code below.

Also, when I compile it, I had warning:
test.c:121: warning: assignment makes pointer from integer without a cast
The line is:
tuple = heap_form_tuple( tupdesc, &dtvalues, &isNull );

Strange.. compiled in a linux box.


If I use: tuple = BuildTupleFromCStrings(attinmeta, values);
it can work but it's string and I want to use bytea.

Also, do I need to free char** values or let postgresql do the job?

Thanks

Billow



/
-- select * from test(1,2,'asdfdsaf') as (id bytea);
CREATE OR REPLACE FUNCTION test(int,int,text)
RETURNS setof record
AS 'gr_indexsearch.so', 'test'
LANGUAGE 'C' IMMUTABLE CALLED ON NULL INPUT;
*/

// PostgreSQL includes
#include "postgres.h"
#include "fmgr.h"

// Tuple building functions and macros
#include "funcapi.h"

#include "utils/builtins.h"


#include 

#define _textout(str) DatumGetPointer(DirectFunctionCall1(textout,
PointerGetDatum(str)))

#ifndef SET_VARSIZE
#define SET_VARSIZE(v,l) (VARATT_SIZEP(v) = (l))
#endif

/* SortMem got renamed in PostgreSQL 8.0 */
#ifndef SortMem
 #define SortMem 16 * 1024
#endif

#ifdef PG_MODULE_MAGIC
PG_MODULE_MAGIC;
#endif


// forward declaration to keep compiler happy
Datum c_complex_add( PG_FUNCTION_ARGS );

PG_FUNCTION_INFO_V1( test );
Datum test( PG_FUNCTION_ARGS )
{
   // things we need to deal with constructing our composite type
   TupleDesc tupdesc;
   HeapTuple tuple;
   Tuplestorestate   *tupstore = NULL;
   ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;

   MemoryContext per_query_ctx;
   MemoryContext oldcontext;

   // Get arguments.  If we declare our function as STRICT, then
   // this check is superfluous.
   if( PG_ARGISNULL(0) ||
   PG_ARGISNULL(1) ||
   PG_ARGISNULL(2))
   {
  PG_RETURN_NULL();
   }

   // Get arguments: TimeStart and TimeEnd
   int32 TimeStart = PG_GETARG_INT32(0);
   int32 TimeEnd   = PG_GETARG_INT32(1);

   // Get Search query
   char *query = _textout(PG_GETARG_TEXT_P(2));


/* check to see if caller supports us returning a tuplestore */
if (!rsinfo || !(rsinfo->allowedModes & SFRM_Materialize))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
 errmsg("materialize mode required, but it is not " \
"allowed in this context")));

/* let the caller know we're sending back a tuplestore */
rsinfo->returnMode = SFRM_Materialize;

per_query_ctx = fcinfo->flinfo->fn_mcxt;
oldcontext = MemoryContextSwitchTo(per_query_ctx);

/* get the requested return tuple description */
tupdesc = rsinfo->expectedDesc;

/* OK, use it */
AttInMetadata *attinmeta = TupleDescGetAttInMetadata(tupdesc);

/* initialize our tuplestore */
tupstore = tuplestore_begin_heap(true, false, SortMem);

char strtest[] = "This is a test!";
int strleng = strlen(strtest);

int rows = 10;
//char** values = (char **) palloc(rows * sizeof(char *));

bytea** values = (bytea **) palloc(rows * sizeof(bytea *));
Datum dtvalues;
bool isNull;

int i;
for(i=0; isetResult = tupstore;

/*
 * SFRM_Materialize mode expects us to return a NULL Datum. The actual
 * tuples are in our tuplestore and passed back through
rsinfo->setResult.
 * rsinfo->setDesc is set to the tuple description that we actually used
 * to build our tuples with, so the caller can verify we did what it was
 * expecting.
 */
rsinfo->setDesc = tupdesc;
MemoryContextSwitchTo(oldcontext);

return (Datum) 0;
}

Re: [HACKERS] Can a C function(server program) be a UDP or TCP server?

[Billow Gao]

Thanks.  This is what I want to know :-)

Regards,

Billow

>Yeah, what he wants is to implement a function in Postgres which does
>something like an LDAP or DNS lookup or something like that.

>Sure you can do this. The only tricky bit is the thing you mentioned about
>reusing the connection. You could always leave the connection in a safe
state
>and don't need to worry about cleaning it up then you could just store it
in a
>static variable which would be the simplest option.

>If you want to use Postgres's facilities for allocating memory and cleaning
it
>up when no longer in use you can use some of the Postgres internal API for
>memory contexts and resource owners. But I don't see any particular reason
you
>should need to worry about this stuff for something simple you're
implementing
>yourself.



On 10/18/07, D'Arcy J.M. Cain <[EMAIL PROTECTED]> wrote:
>
> On Thu, 18 Oct 2007 11:24:24 -0400
> "Billow Gao" <[EMAIL PROTECTED]> wrote:
> > I can write the network program.
> > But I am not 100% sure whether I can add the c-language function (
> > http://www.postgresql.org/docs/8.2/interactive/xfunc-c.html)
> > to PostgreSQL. The function will be dynamic loaded by PostgreSQL.
> > I want to know whether there are any limitation on the function I wrote.
> >
> > for example:
> > If I want to write a function:
> >
> > PG_FUNCTION_INFO_V1(c_talktoremoteudp);
> >
> >
> > And use it in PostgreSQL like:
> >
> > =
> > SELECT name, c_talktoremoteudp
> >
> > (emp, 1500) AS overpaid
> > FROM emp
> > WHERE name = 'Bill' OR name = 'Sam';
> >
> > =
> > The function c_talktoremoteudp will:
> > 1. send udp data to remote udp server
> > 2. monitor an udp port and wait for the reply
> > 3. return the data to the select query.
>
> I am confused.  The dynamic function resides in the server.  The query
> runs in the server.  Where is the "remoteness" in any of this?  Are you
> saying that there is a second server that is not PostgreSQL that uses
> UDP that you want to communicate with and merge info into the
> PostgreSQL server from?
>
> --
> D'Arcy J.M. Cain <[EMAIL PROTECTED]> |  Democracy is three wolves
> http://www.druid.net/darcy/|  and a sheep voting on
> +1 416 425 1212 (DoD#0082)(eNTP)   |  what's for dinner.
>
> ---(end of broadcast)---
> TIP 2: Don't 'kill -9' the postmaster
>

Re: [HACKERS] Can a C function(server program) be a UDP or TCP server?

[Billow Gao]

On Thu, 18 Oct 2007 10:55:19 -0400
"Billow Gao" <[EMAIL PROTECTED]> wrote:
> Is it possible to write a dynamic loaded C function as an UDP or TCP
server?
>
> What we want to do it is:
> Add a search function which send a UDP package to remote UDP server
> and then listen to an UDP port, waiting for the result.
> Ideally, we don't close the UDP server after the search query end.
> So we can reuse it for next search.
>
> Is it possible?

>Short answer: yes.  Slightly longer answer: If you need to ask this
>quetion then you should really talk to someone about network
>programming but this is the wrong list.

Thanks for your reply.

I can write the network program.
But I am not 100% sure whether I can add the c-language function (
http://www.postgresql.org/docs/8.2/interactive/xfunc-c.html)
to PostgreSQL. The function will be dynamic loaded by PostgreSQL.
I want to know whether there are any limitation on the function I wrote.

for example:
If I want to write a function:

PG_FUNCTION_INFO_V1(c_talktoremoteudp);


And use it in PostgreSQL like:

=
SELECT name, c_talktoremoteudp

(emp, 1500) AS overpaid
FROM emp
WHERE name = 'Bill' OR name = 'Sam';

=
The function c_talktoremoteudp will:
1. send udp data to remote udp server
2. monitor an udp port and wait for the reply
3. return the data to the select query.

Anyway, I guess that it can be done. I will write a simple function to test
it.

Ying


>If you are asking if PostgreSQL already does UDP then the answer is
>no.  You need to write a server program that talks UDP in one direction
>and TCP to PostgreSQL in the other direction.  Watch out for security
>issues.

[HACKERS] Can a C function(server program) be a UDP or TCP server?

[Billow Gao]

Hi there,

Is it possible to write a dynamic loaded C function as an UDP or TCP server?

What we want to do it is:
Add a search function which send a UDP package to remote UDP server
and then listen to an UDP port, waiting for the result.
Ideally, we don't close the UDP server after the search query end.
So we can reuse it for next search.

Is it possible?

Thanks

Billow