Re: [HACKERS] HISTIGNORE for psql
Hello, On Fri, 18 Aug 2017, Vesa-Matti J Kari wrote: > A quick patch is attached. Not sure about the quality, hacked this > together in about four hours, trying to figure out how to do it correctly > the PostgreSQL way. Sorry, I guess I have written too many Python scripts. I no longer remember to free() memory in C, at least tokbuf should be free():d before returning in parse_histignore(). Regards, vmk -- Tietotekniikkakeskus / Helsingin yliopisto IT department / University of Helsinki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] HISTIGNORE for psql
Hello,
On Thu, 17 Aug 2017, Pavel Stehule wrote:
> 2017-08-17 9:23 GMT+02:00 Vesa-Matti J Kari <vmk...@cc.helsinki.fi>:
>
> Bash has HISTIGNORE feature that allows you to exclude certain commands
> from the command history (e.g. shutdown, reboot, rm *).
>
> Would it make any sense to add such a feature to psql (e.g. to ignore
> DROP, DELETE commands)?
>
>
> It is not bad idea.
A quick patch is attached. Not sure about the quality, hacked this
together in about four hours, trying to figure out how to do it correctly
the PostgreSQL way.
Based on a few tests, the patch seems to work.
I do not know how the Bash implementation works, but I chose to disallow
forms such as:
:
:a
a:
a::b
So specifying empty strings seems like a syntax error to me. But I do
not know how to report failures for those, the current patch disallows
them and HISTIGNORE simply does not work with invalid syntax.
Regards,
vmk
--
Tietotekniikkakeskus / Helsingin yliopisto
IT department / University of Helsinki
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index b3dbb5946e..1a28a21f26 100644
*** a/src/bin/psql/help.c
--- b/src/bin/psql/help.c
***
*** 357,362 helpVariables(unsigned short int pager)
--- 357,363
" (default:
0=unlimited)\n"));
fprintf(output, _(" HISTCONTROLcontrols command history
[ignorespace, ignoredups, ignoreboth]\n"));
fprintf(output, _(" HISTFILE file name used to store the
command history\n"));
+ fprintf(output, _(" HISTIGNORE controls command history,
ignores colon separated commands\n"));
fprintf(output, _(" HISTSIZE max number of commands to store
in the command history\n"));
fprintf(output, _(" HOST the currently connected
database server host\n"));
fprintf(output, _(" IGNOREEOF number of EOFs needed to
terminate an interactive session\n"));
diff --git a/src/bin/psql/index 62f5f77383..ffaee397cf 100644
*** a/src/bin/psql/input.c
--- b/src/bin/psql/input.c
***
*** 147,152 pg_send_history(PQExpBuffer history_buf)
--- 147,162
if (useHistory && s[0])
{
+ if (pset.histignore)
+ {
+ for (i = 0; i < histignore.nstr; i++) {
+ if (pg_strncasecmp(s, histignore.str[i],
strlen(histignore.str[i])) == 0) {
+ resetPQExpBuffer(history_buf);
+ return;
+ }
+ }
+ }
+
if (((pset.histcontrol & hctl_ignorespace) &&
s[0] == ' ') ||
((pset.histcontrol & hctl_ignoredups) &&
diff --git a/src/bin/psql/sindex b78f151acd..9c436eeb88 100644
*** a/src/bin/psql/settings.h
--- b/src/bin/psql/settings.h
***
*** 77,82 enum trivalue
--- 77,89
TRI_YES
};
+ typedef struct HistIgnore
+ {
+ char **str;
+ int nstr;
+
+ } HistIgnore;
+
typedef struct _psqlSettings
{
PGconn *db; /* connection to backend */
***
*** 133,138 typedef struct _psqlSettings
--- 140,146
PSQL_ERROR_ROLLBACK on_error_rollback;
PSQL_COMP_CASE comp_case;
HistControl histcontrol;
+ char*histignore;
const char *prompt1;
const char *prompt2;
const char *prompt3;
***
*** 141,146 typedef struct _psqlSettings
--- 149,155
} PsqlSettings;
extern PsqlSettings pset;
+ extern HistIgnore histignore;
#ifndef EXIT_SUCCESS
diff --git a/src/bin/psql/starindex 7f767976a5..b14ce2699e 100644
*** a/src/bin/psql/startup.c
--- b/src/bin/psql/startup.c
***
*** 31,36
--- 31,37
* Global psql options
*/
PsqlSettings pset;
+ HistIgnore histignore;
#ifndef WIN32
#define SYSPSQLRC "psqlrc"
***
*** 667,672 parse_psql_options(int argc, char *argv[], struct adhoc_opts
*options)
--- 668,726
}
+ static bool
+ parse_histignore(const char *val)
+ {
+ const char *p;
+ char *tokbuf;
+ char *tok;
+ int i;
+ int len;
+ int ncolons;
+
+ len = strlen(val);
+
+ if (len == 0) {
+ histignore.str = NULL;
+ histignore.nstr = 0;
+ return true;
+ }
+
+ /* Validate syntax first. */
+ if (val[0] == ':' || val[len-1] == ':')
+ return f
[HACKERS] HISTIGNORE for psql
Hello, Bash has HISTIGNORE feature that allows you to exclude certain commands from the command history (e.g. shutdown, reboot, rm *). Would it make any sense to add such a feature to psql (e.g. to ignore DROP, DELETE commands)? Regards, vmk -- Tietotekniikkakeskus / Helsingin yliopisto IT department / University of Helsinki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Strange hanging bug in a simple milter
Hello, On Mon, 23 Sep 2013, Stephen Frost wrote: I've now committed a fix for this issue. I cloned the 9.4devel branch and linked my authmilter and a test program (based on Heikki's earlier design) against the libpq that comes with it. After hours of pretty extensive stress testing using 2, 3, 4, 5, 6, 50, 100 and 500 parallel threads, I have observed no deadlocks. Everything runs smoothly. Many thanks to all who contributed to the fix. Regards, vmk -- Tietotekniikkakeskus / Helsingin yliopisto IT department / University of Helsinki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Strange hanging bug in a simple milter
Hello PostgreSQL gurus, (I have already posted a very similar message to comp.mail.sendmail newsgroup on August 22nd, but I haven't received any responses there. I have also tried pgsql-interfa...@postgresql.org but to no avail. Solving this problem requires some Sendmail/Postfix experience because the MTA needs to be configured to use the authmilter. Also some basic Milter API knowledge is needed: https://www.milter.org/developers/api/index) I have come across a very strange bug and I do not understand why it is occurring. Fortunately it is reproducible in a pretty deterministic manner. The goal: to create a milter for limiting how many nrcpts an authenticated user can send in a 24 hour time interval. The related data is stored in a PostgreSQL database. The problem: my alpha-stage code for authmilter simply hangs when processing two concurrent connections from Sendmail. For authmilter-simplified, I have removed the callbacks envfrom and envrcpt, because they are not needed in order to demonstrate the bug. Basically all that the authmilter now does is to connect to PostgreSQL in authmilt_connect() and close the connection in authmilt_close(). Based on the authmilter debug logging it seems to me that when the hanging occurs, the authmilter never completes PQsetdbLogin(). Based on the document http://www.postgresql.org/docs/9.1/interactive/libpq-threading.html I am sure libpq should be thread safe and on startup the authmilter verifies that the libpq is indeed thread safe. If you think you could set up a test enviroment using: - Sendmail (or maybe Postfix) - authmilter - PostgreSQL here is an authmilter-simplied.tar.gz package for you: http://www.helsinki.fi/~vmkari/authmilter-simplified.tar.gz The README file contains a rough instructions outline on how to setup things in order to reproduce the strange hanging bug. Please note that when running two test message sender scripts in parallel, the bug does not occur immediately, but only after between 1 to 5 minutes of processing. Sometimes it may take even longer. I have tested authmilter on Ubuntu Linux 12.04 having packages: libmilter-dev 8.14.4-2ubuntu2 libmilter1.0.1 8.14.4-2ubuntu2 libpq-dev 9.1.9-0ubuntu12.04 libpq5 9.1.9-0ubuntu12.04 postgresql-9.1 9.1.9-0ubuntu12.04 postgresql-client-9.1 9.1.9-0ubuntu12.04 postgresql-client-common 129ubuntu1 postgresql-common 129ubuntu1 postgresql-contrib-9.1 9.1.9-0ubuntu12.04 postgresql-doc-9.1 9.1.9-0ubuntu12.04 postgresql-server-dev-9.1 9.1.9-0ubuntu12.04 postgresql-server-dev-all 129ubuntu1 sendmail 8.14.4-2ubuntu2 sendmail-base 8.14.4-2ubuntu2 sendmail-bin 8.14.4-2ubuntu2 sendmail-cf 8.14.4-2ubuntu2 sendmail-doc 8.14.4-2ubuntu2 I suspect there could be something wrong with libpq and libmilter working together, but I am not sure. Many thanks for any help you can provide. PS. I installed PostgreSQL 9.3rc1 and linked my milter against the libpq that comes with that version. The hanging bug still occurs. Regards, vmk -- Tietotekniikkakeskus / Helsingin yliopisto IT department / University of Helsinki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Strange hanging bug in a simple milter
Hello, On Mon, 9 Sep 2013, Heikki Linnakangas wrote: I managed to set that up and got it running. Many thanks for taking the time. But it works fine for me, does not hang. Okay. Have you tried increasing the iterations for the smtp sender scripts? And could you please specify what is your test environment like (i.e. OS and the related library versions)? I'd suggest poking around with gdb, to see where it hangs. I have actually done that, but it only show the main listener thread from the libmilter library: (gdb) bt #0 0x7fe64bdd0313 in poll () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x7fe64c4f7b46 in mi_listener () from /usr/lib/libmilter.so.1.0.1 #2 0x7fe64c4f8707 in smfi_main () from /usr/lib/libmilter.so.1.0.1 #3 0x00402c8f in main (argc=15, argv=0x7fffa6560e68) at authmilter.c:699 Hmmm. The man page mentioned no threads, but Google was helpful and suggested info threads so here goes: (I hope alpine will not wrap these long lines) (gdb) info threads Id Target Id Frame 9Thread 0x7fe64700c700 (LWP 14362) authmilter 0x7fe64c0b69f7 in do_sigwait () from /lib/x86_64-linux-gnu/libpthread.so.0 8Thread 0x7fe64680b700 (LWP 14363) authmilter 0x7fe64bdd0313 in poll () from /lib/x86_64-linux-gnu/libc.so.6 7Thread 0x7fe645809700 (LWP 14365) authmilter 0x7fe64c0b589c in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0 6Thread 0x7fe645008700 (LWP 22404) authmilter 0x7fe64c0b589c in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0 5Thread 0x7fe64600a700 (LWP 27263) authmilter 0x7fe64c0b589c in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0 4Thread 0x7fe644807700 (LWP 27264) authmilter 0x7fe64c0b589c in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0 3Thread 0x7fe62700 (LWP 27283) authmilter 0x7fe64c0b589c in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0 2Thread 0x7fe62f7fe700 (LWP 27284) authmilter 0x7fe64c0b589c in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0 * 1Thread 0x7fe64c8fd740 (LWP 14361) authmilter 0x7fe64bdd0313 in poll () from /lib/x86_64-linux-gnu/libc.so.6 It looks like a deadlock situation of some kind... (gdb) thread 2 [Switching to thread 2 (Thread 0x7fe62f7fe700 (LWP 27284))] #0 0x7fe64c0b589c in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0 (gdb) bt #0 0x7fe64c0b589c in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0 #1 0x7fe64c0b1065 in _L_lock_858 () from /lib/x86_64-linux-gnu/libpthread.so.0 #2 0x7fe64c0b0eba in pthread_mutex_lock () from /lib/x86_64-linux-gnu/libpthread.so.0 #3 0x7fe64c2df200 in ?? () from /usr/lib/libpq.so.5 #4 0x7fe64b78a5f5 in ?? () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 #5 0x7fe64b77a915 in RSA_new_method () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 #6 0x7fe64b77d64d in ?? () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 #7 0x7fe64b7b9bf2 in ?? () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 #8 0x7fe64b7bc6d1 in ASN1_item_ex_d2i () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 #9 0x7fe64b7bd0c4 in ASN1_item_d2i () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 #10 0x7fe64b77ea2f in ?? () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 #11 0x7fe64b7b461a in X509_PUBKEY_get () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 #12 0x7fe64b7d119a in X509_get_pubkey_parameters () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 #13 0x7fe64b7d1398 in X509_verify_cert () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 #14 0x7fe64bac52f8 in ?? () from /lib/x86_64-linux-gnu/libssl.so.1.0.0 #15 0x7fe64baa2ef3 in ?? () from /lib/x86_64-linux-gnu/libssl.so.1.0.0 #16 0x7fe64baa7222 in ?? () from /lib/x86_64-linux-gnu/libssl.so.1.0.0 #17 0x7fe64c2dffcb in ?? () from /usr/lib/libpq.so.5 #18 0x7fe64c2d0c5e in PQconnectPoll () from /usr/lib/libpq.so.5 #19 0x7fe64c2d1e3e in ?? () from /usr/lib/libpq.so.5 #20 0x7fe64c2d26f8 in PQsetdbLogin () from /usr/lib/libpq.so.5 #21 0x00401ba5 in authmilt_connect (ctx=0xe81b60, hostname=0x7fe628c0 localhost, hostaddr=0x7fe62f7fdce0) at authmilter.c:212 #22 0x7fe64c4f69dc in ?? () from /usr/lib/libmilter.so.1.0.1 #23 0x7fe64c4f5f5f in mi_engine () from /usr/lib/libmilter.so.1.0.1 #24 0x7fe64c4fada6 in ?? () from /usr/lib/libmilter.so.1.0.1 #25 0x7fe64c0aee9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #26 0x7fe64bddbccd in clone () from /lib/x86_64-linux-gnu/libc.so.6 #27 0x in ?? () (gdb) thread 3 [Switching to thread 3 (Thread 0x7fe62700 (LWP 27283))] #0 0x7fe64c0b589c in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0 (gdb) bt #0 0x7fe64c0b589c in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0 #1 0x7fe64c0b1065 in _L_lock_858 () from
