Re: [GENERAL] Column Tetris Automatisation

2017-01-15 Thread Mike Sofen
does matter...it all ties together nicely, making it easier for other developers to follow an identical pattern across all of the database objects. All of that said, the notion of embedding Tetris functionality into a codebase makes me smile, for some reason... Mike Sofen -- Sent via pgsql-

Re: [GENERAL] LYDB: What advice about stored procedures and other server side code?

2016-12-28 Thread Mike Sofen
ORMs are a shortcut to getting an app talking to data, but aren't a substitute for a proper, scalable data tier. IMO...being a data specialist... :-) Mike Sofen (Synthetic Genomics) -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http:

Re: [GENERAL] Performance PLV8 vs PLPGSQL

2016-12-28 Thread Mike Sofen
tailored the model to match. Mike Sofen (Synthetic Genomics)

Re: [GENERAL] Syntax error needs fresh eyeballs

2016-12-26 Thread Mike Sofen
From: Rich Shepard My schema includes three tables and psql throws an error I'm not seeing when I try to read the schema into the database. I'd appreciate fresh eyes looking at the table and learning what error I'm not seeing. The tables: CREATE TABLE Weather ( site_id INTEGER

Re: [GENERAL] Storing files: 2.3TBytes, 17M file count

2016-11-28 Thread Mike Sofen
, this is all real time web app stuff. This is a model that could work for anyone dealing with large objects (text or binary). The nice part is, the original 25TB of data storage drops to 5TB – a much more manageable number, allowing for significant growth, which is on the horizon. Mike

Re: [GENERAL] initdb createuser commands

2016-10-30 Thread Mike Sofen
led in sql, the requests you’ve made won’t assist them at all. Mike Sofen (Synthetic Genomics)

Re: [GENERAL] Immutable datastore library?

2016-10-18 Thread Mike Sofen
re small - up to 100k rows or less, and processed in under 100ms. But even when there was a network outage and we had to do a catch up load with millions of rows, it ran very quickly. IOWs, the double write overhead was very modest, especially with modern disk performance. Mike Sofen (Synthetic Genomics)

Re: [GENERAL] Graphical entity relation model

2016-09-28 Thread Mike Sofen
), and the fully comprehensive, with a very modern looking UI. In contrast, there are the over-priced dinosaurs with old ugly UIs. A while back I reviewed some of the modeling tools, and none did it for me, I went ahead and got another license to xcase. Mike Sofen (Synthetic Genomics)

Re: [GENERAL] how to monitor the progress of really large bulk operations?

2016-09-28 Thread Mike Sofen
From: Pavel StehuleSent: Tuesday, September 27, 2016 9:18 PM 2016-09-28 6:13 GMT+02:00 Pavel Stehule <pavel.steh...@gmail.com <mailto:pavel.steh...@gmail.com> >: Hi 2016-09-27 23:03 GMT+02:00 Mike Sofen <mso...@runbox.com <mailto:mso...@runbox.com> >: Hi gang

[GENERAL] how to monitor the progress of really large bulk operations?

2016-09-27 Thread Mike Sofen
to view the state of a transaction in flight, seeing how many rows have been read or inserted (possible for a transaction in flight?), memory allocations across the various PG processes, etc. Possible or a hallucination? Mike Sofen (Synthetic Genomics)

Re: [GENERAL] IDE for function/stored proc development.

2016-09-03 Thread Mike Sofen
From: Tim Uckun Sent: Saturday, September 03, 2016 2:37 AM Does anybody use an IDE for doing heavy duty stored proc development? PGadmin is decent but I am looking for something better. I have been using the Datagrip app (from Jetbrains), from its beta release up through now v 2016.2 and

Re: [GENERAL] Rackspace to RDS using DMS (Postgres 9.2)

2016-09-01 Thread Mike Sofen
From: Joshua D. Drake Sent: Thursday, September 01, 2016 9:29 AM On 08/31/2016 03:41 PM, Patrick B wrote: > Hi guys, I posted this question on the ADMIN list but will post here > as well so more people can comment... >

Re: [GENERAL] UPDATE OR REPLACE?

2016-09-01 Thread Mike Sofen
ings. That doesn't mean that they should. Mike Sofen (USA) -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] UUIDs & Clustered Indexes

2016-08-30 Thread Mike Sofen
From: George Neuner Sent: Tuesday, August 30, 2016 5:54 PM >Mike Sofen wrote: So in this scenario, I'm using >BOTH bigserials as the PK and uuids as AKs in the core tables. I >reference the bigints for all joins and (have to) use the uuids for the >filters. It's been worki

Re: [GENERAL] UUIDs & Clustered Indexes

2016-08-30 Thread Mike Sofen
in case I need to shard, since I'll need/want that uniqueness across servers. Mike Sofen

Re: [GENERAL] Re: Clustered index to preserve data locality in a multitenant application?

2016-08-30 Thread Mike Sofen
r loaner Cisco UCS server. Should have that posted to the Perform list later this week. Mike Sofen (USA)

Re: [GENERAL] Stored procedure version control

2016-06-30 Thread Mike Sofen
out leaving the editor. My coding efficiency using this model is quite high...the overhead of using git is trivial. For rollbacks, we can simply point to the prior stored proc version and recompile those. For DDL rollbacks, I have to code those scripts and supply them...this is the one place I hav

Re: [GENERAL] OT hardware recommend

2016-06-18 Thread Mike Sofen
From: Adam Brusselback Sent: Saturday, June 18, 2016 1:26 PM Subject: Re: [GENERAL] OT hardware recommend It's really amazing how much solid state drives transferred the database bottleneck away from disk. Adam – so very true. We used to spend ungodly amounts of time/money/effort to

Re: [GENERAL] PostgresSQL and HIPAA compliance

2016-06-17 Thread Mike Sofen
-Original Message- From: Alex John Sent: Friday, June 17, 2016 3:04 AM To: pgsql-general@postgresql.org Subject: [GENERAL] PostgresSQL and HIPAA compliance Hello, I have a few questions regarding the use of PostgreSQL and HIPAA compliance. I work for a company that plans on storing

Re: [GENERAL] regarding schema only migration from sqlserver to postgres with runmtk.sh

2016-06-15 Thread Mike Sofen
From: pgsql-general-ow...@postgresql.org [mailto:pgsql-general-ow...@postgresql.org] On Behalf Of David G. Johnston Sent: Wednesday, June 15, 2016 1:31 PM To: Durgamahesh Manne Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] regarding schema only migration

Re: [GENERAL] Alternate or Optimization for with hold cursor

2016-05-12 Thread Mike Sofen
stantly. And since you're working on a single row at time, it will take forever. Convert the cursor into a normal query and you should see BIG (10-1000x) gains in speed. A cursor can always be converted to normal sql...always...it's not always easy but it's always worth the effort. Mike Sofen

Re: [GENERAL] Thoughts on "Love Your Database"

2016-05-04 Thread Mike Sofen
nctionalities like windowing functions. So I guess you’d say I’m in the entirely opposite camp, since it’s proven to be such an effective solution architecture for many applications that leverage relational database engines. Mike Sofen (San Diego, CA USA)

Re: [GENERAL] Thoughts on "Love Your Database"

2016-05-04 Thread Mike Sofen
olerance for slack db design that could cause scalability or performance issues. My stored functions are...relatively simple. Mike Sofen (San Diego, CA USA) -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Function PostgreSQL 9.2

2016-05-03 Thread Mike Sofen
lose interest in collaborating with them, if you catch my drift… Mike Sofen

Re: [GENERAL] (VERY) Slow Query - PostgreSQL 9.2

2016-05-03 Thread Mike Sofen
decomposed from the long full_path string via post-processing after the insert. A third consideration would be to archive off older/unneeded rows to a history table to reduce row counts. This is about proper structure. Mike Sofen

Re: [GENERAL] truncate table getting blocked

2016-04-26 Thread Mike Sofen
From: Jayadevan M Sent: Tuesday, April 26, 2016 6:32 AM Hello, I have a python script. It opens a cursor… Thanks, Jayadevan

Re: [GENERAL] Function PostgreSQL 9.2

2016-04-19 Thread Mike Sofen
This is such a poorly designed, hacked together “thing” – it isn’t a database, it’s someone’s idea of how to store data when they don’t know how to store data, like they moved it from Access or Excel. Just start over and design a proper relational schema with best practices and you’ll

Re: [GENERAL] Re: Why is the comparison between timestamp and date so much slower then between two dates

2016-04-14 Thread Mike Sofen
stead I see people using functions within functions within functions, the cascading impact of which becomes very hard to unravel. Mike Sofen -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] I/O - Increase RAM

2016-04-13 Thread Mike Sofen
|From: John R Pierce Sent: Wednesday, April 13, 2016 1:53 PM | |On 4/13/2016 1:43 PM, drum.lu...@gmail.com wrote: |> At the moment I'm having 100% I/O during the day. My server has SATA |> HDs, and it can't be changed now. |> So, to solve the problem (or at least try) I was thinking about double

Re: [GENERAL] How to delete few elements from array beginning?

2016-03-09 Thread Mike Sofen
>>Alexander Farber wrote on Wednesday, March 09, 2016 4:11 AM Hello fellow PostgreSQL users, what is please the most efficient way to delete a slice from the start of a longer array (after I have copied it to another array)? Do I really have to copy a large slice of the array to itself,