Object fields (Was: Re: QUERY BY ATTRIBUTE is slow af)

[Bernd Fröhlich via 4D_Tech]

David Adams:

> I've been asking about real-world use of object fields for quite some time

Here´s my real-world use:
I wrote an iPhone app that collects data (scans barcodes).
That data is in JSON format and transferred via HTTP to a 4D Server.
The server stores that data in an object field (which is never queried. It just 
holds the collected data).
The beauty is: when I need to send additional data that I did not think of 
while writing the app, I just stuff that data into the JSON along with the rest.
Of course the parsing routine on 4D Server has to be tweaked to make use of the 
additional data, but that´s all that is needed.
I am quite happy with this approach.
The same would work with a text- or blob-field to store the data, but an object 
field makes it a bit easier.

Greetings from Germany,
Bernd Fröhlich
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: QUERY BY ATTRIBUTE is slow af

[John DeSoi via 4D_Tech]

> On Sep 12, 2017, at 5:48 PM, David Adams via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> Exciting stuff. I've read a ton about it and done some tests. Rob's plugin
> doesn't have an explicit command for transferring object fields, but you
> can stringify them and send them as text.

Most Postgres client libraries use text rather than binary encoding because the 
binary formats are not documented. Of course, you have the source but that is a 
lot of work :). But the binary option is especially fast and useful when you 
are sending/receiving binary, aka Postgres bytea. 

https://stackoverflow.com/questions/35600070/postgres-jsonb-specification-for-copy-in-binary-format


> If you have exotic searching requirements, Postgres has a ton of
> engine-level features...and a common add-on gives you a bunch of fuzzy
> matching tools. I was talking with John DeSoi the other day and he says
> that the fuzzy tools add-on includes "tri-grams". That's "n-grams" or
> "q-grams" with a length of 3. I've got a stack of research papers on why
> trigrams are so frickin amazing at fuzzy matching. I've tried (and failed)
> to implement them effectively in 4D a couple of times. WIth Postgres, you
> just add the fuzzy tools and away you go.

Here are a couple of links if anyone is interested.

https://www.postgresql.org/docs/9.6/static/pgtrgm.html

https://www.postgresql.org/docs/9.6/static/textsearch.html

John DeSoi, Ph.D.


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: QUERY BY ATTRIBUTE is slow af

[David Adams via 4D_Tech]

On Wed, Sep 13, 2017 at 9:25 AM, Tim Nevels via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> On Sep 12, 2017, at 5:49 PM, Jeffrey Kain wrote:
>
> > Yeah, so many caveats to object fields. Bite the bullet, use a related
> table...
>
> Remember 4D’s implementation strategy for new features:
>
> - You get a little bit at a time.
> - The feature set is not complete.
> - The public (i.e. 4D developers) are not provided with any information
> about what the completed feature set will ultimately look like.
>

Nicely summarized.

About 20 years ago, I decided "I won't base any of my choices on software
that isn't shipping." (Fool me 18 times, shame on you, fool me 19 times,
shame on me.)  I might bend that a bit for a company with a detailed
roadmap, and a really solid record of delivering on that roadmap. Even
then, I wouldn't roll the dice on something that would be a huge disaster
if it didn't ship on time or with the feature as expected. I've never, ever
regretted this choice. This isn't specifically about 4D by any means, but
they aren't excepted from the rule either.

With 4D, we have no road map so the *only* features and behaviors we can
plan on are the ones in our hands *today.* That's fine, most of the time I
only want to know how things work now so that I can make plans. 4D is under
no obligation to provide whatever pet feature requests any of us have.
Clearly. Still, when I hear about "future versions" I file them under "who
cares?" I mean, it's fun to think about - but it does nothing for my work
today and does nothing to my plans.

It's like when the medical world says that something will be a viable
treatment in "5-10" years. That's pure science fiction, they have no idea
how it will play out when something is that far out. So, fun to think
about, but not useful for making choices.

So, yeah, object fields are already useful in a limited number of cases,
IMO, and may evolve into something far more powerful and useful. Let's hope
so! But for now, when I need more exotic/performant/scalable features,
there's Postgres where all of this stuff already has years of field use.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Connecting to SQL on a PC

[Charles Miller via 4D_Tech]

The same way, except the drivers are already there. There is an ODBC admin 
function similar to Mac
Log in could be different depending on how access is set up.

Regards

Chuck

 Chuck Miller Voice: (617) 739-0306
 Informed Solutions, Inc. Fax: (617) 232-1064   
mailto:miller.cjaygmail.com 
 Brookline, MA 02446 USA Registered 4D Developer
   Providers of 4D and Sybase connectivity
  http://www.informed-solutions.com  



> On Sep 12, 2017, at 6:38 PM, Robert McKeever via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
> For years, we have had a couple of clients that connect to an SQL server. All 
> the traffic is done using triggers under 4D Server (the triggers run on the 
> server), and we use the Actual OBDC SQL drivers for the connection on a Mac 
> server. Works well.
> 
> Now I have received a request to draw some data back to print on a surgical 
> booking form. I assume that the PC wanting to print will need to request the 
> data. So, how does one connect to an SQL server from a PC?
> 
> _
> Bob McKeever  http://www.mswl.com 
> McKeever's Software Wizardry
> Port Coquitlam, B.C.
> bobmckee...@mac.com
> 
> 
> 
> 
> **
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D and Fusion Charts

[Jeremy French via 4D_Tech]

Have you looked at these items in the Web Area documentation?

1) Configuring the Web area
2) Using the $4d object

See Accessing 4D Methods at
http://doc.4d.com/4Dv16/4D/16.2/Programmed-management-of-Web-Areas.300-3434143.en.html#3190797



> On Sep 12, 2017, at 2:53 PM, John S. Poteat via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
> What I would like to have happen now is if I click on an element in the Chart 
> (Ex: Bar or Pie Piece)
> to make a call back to 4D and have my selection of records change to show the 
> records that 
> correspond to the Chart element that was clicked on.

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: QUERY BY ATTRIBUTE is slow af

[Tim Nevels via 4D_Tech]

On Sep 12, 2017, at 5:49 PM, Jeffrey Kain wrote:

> Yeah, so many caveats to object fields. Bite the bullet, use a related 
> table...

Remember 4D’s implementation strategy for new features:

- You get a little bit at a time. 
- The feature set is not complete.
- The public (i.e. 4D developers) are not provided with any information about 
what the completed feature set will ultimately look like. 

It’s quite possible 4D Engineering doesn’t even know what the full feature set 
of object fields will be. Could be they just keep adding features based on what 
developers say we need, or maybe what 4D Engineering “thinks” a developer might 
need or find useful. 

Add to this the fact that 4D Engineers are not, and have never been, 4D 
developers. They don’t really use the thing that they are creating. 

Don’t get me wrong, I know 4D Inc. uses 4D internally for a lot of things. So 
4D Inc. does eat their own dog food. But the guys in the kitchen making the dog 
food never really taste it. They cook it and then send it off for others to eat 
and wait to hear how good it tastes. And if it need more salt. Or if it is too 
spicy. You know what I’m saying.

Tim


Tim Nevels
Innovative Solutions
785-749-3444
timnev...@mac.com


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D and Fusion Charts

[Tom DeMeo via 4D_Tech]

Hi John,


Fusion charts will allow you to set an on click event handler in javascript. 
You need that function to be part of the page you are loading in the web area. 
You probably can look at one of your Wakanda examples to see how that event 
handler is written. Inside that function, you can make a call back to 4D, using 
the $4d object. For a detailed explanation, take a look at the following 
documentation:


http://doc.4d.com/4Dv16/4D/16.2/Programmed-management-of-Web-Areas.300-3434143.en.html
 



About half way down that page is a good description of how to access 4D methods 
and write your function. 

Look for “Accessing 4D methods” and “Using the $4d object”


___


Tom DeMeo





**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: QUERY BY ATTRIBUTE is slow af

[David Adams via 4D_Tech]

That's a good article, if you want more - ping me. And that's exactly the
idea I was referring to when I said I'd be willing to learn some new
operators if it bought me a lot. And, since I tend to write like I know
what I'm talking about no matter if I do or don't ;-) At the least, John
DeSoi and Jim Crate on this list seem to *really* know what they're talking
about with regards to Postgres.

Yeah, Postgres has several ways of dealing with JSON. You can use a plain
JSON field if all you want to do is pipeline or store some JSON for some
reason (pre-rendered values for pulls, logged API calls for review, etc.)
JSONB is a lot different and takes advantage of the GIN index. From the
sound of it, the GIN index is the real genius of the JSONB type. GIN has
been in development for over a decade and helps provide Solr-like search
features. (Finding words based on how near they are to other words, for
example.)

For space, JSON fields are smaller than comparable 4D object fields because
they don't default to UTF16 and can be configured to use a much smaller
collation. Assuming you can it your data into a smaller character
encoding/set/whatever you call it. JSONB isn't space-optimized by default,
but you can get an add-on to improve storage by a factor of roughly 6:1.
But, really, the reason to use JSONB instead of JSON is for the searching.
If you just need to store JSON, a JSON field is fine. (A text field is also
fine, but a JSON field gives you database-level validation that the string
*is* JSON.)

Exciting stuff. I've read a ton about it and done some tests. Rob's plugin
doesn't have an explicit command for transferring object fields, but you
can stringify them and send them as text. The default rules for escaping
text in Postgres are easy:

-- Convert single quotes into two single quotes.
-- Put single quotes around the whole block.

If you have exotic searching requirements, Postgres has a ton of
engine-level features...and a common add-on gives you a bunch of fuzzy
matching tools. I was talking with John DeSoi the other day and he says
that the fuzzy tools add-on includes "tri-grams". That's "n-grams" or
"q-grams" with a length of 3. I've got a stack of research papers on why
trigrams are so frickin amazing at fuzzy matching. I've tried (and failed)
to implement them effectively in 4D a couple of times. WIth Postgres, you
just add the fuzzy tools and away you go.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Connecting to SQL on a PC

[Robert McKeever via 4D_Tech]

For years, we have had a couple of clients that connect to an SQL server. All 
the traffic is done using triggers under 4D Server (the triggers run on the 
server), and we use the Actual OBDC SQL drivers for the connection on a Mac 
server. Works well.

Now I have received a request to draw some data back to print on a surgical 
booking form. I assume that the PC wanting to print will need to request the 
data. So, how does one connect to an SQL server from a PC?

_
Bob McKeever  http://www.mswl.com 
McKeever's Software Wizardry
Port Coquitlam, B.C.
bobmckee...@mac.com




**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: QUERY BY ATTRIBUTE is slow af

[Jeffrey Kain via 4D_Tech]

I agree.

I had just read this article the other day, however, and was hoping (expecting) 
that 4D would provide something similar to Postgres JSONB and GIN indexes:

https://compose.com/articles/is-postgresql-your-next-json-database/

--
Jeffrey Kain
jeffrey.k...@gmail.com

> On Sep 12, 2017, at 5:39 PM, David Adams via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> Seriously people, 4D's hostility to the relational model has been around
> since longer than I have, and it makes no sense. The beauty of normalized
> data is that you can easily answer *unanticipated questions.* Also, you can
> store your information in a way that's easy to keep in order. The problem
> with subtables wasn't just the implementation of the idea, it was the idea
> itself. Object fields for relational data? Much the same.

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: QUERY BY ATTRIBUTE is slow af

[David Adams via 4D_Tech]

Jeff,

Well, thanks for trying to be the cranky old man instead of me. But just
won't do. Now I have to rant.

My skepticism regarding using object fields for the purposes should be
pretty obvious to anyone that's paid attention.

Regarding the Summit demo...I watched the video and thought that it was
very well presented and that it took a ton of work to do. I also think that
Olivier is a very nice person personally and professionally. It was a pity
that there was effectively no Q & A. Two points from the presentation:

1) The "optimization" from the object field depended on knowing the *exact*
question(s) that you wanted answered in advance. Unanticipated question?
Then the object search might end up being sequential.

2) The overall thrust of the demo seemed to be to show that object fields
represent some kind of a sweet spot between a one table, one field database
and a fully normalized database. The claim being that full normalization is
slower and harder to do. Hmmm. Didn't make sense. I downloaded the demo,
looked at the relational joining code, rewrote it to something more
sensibleand the so-called speed "optimization" from object fields went
away. I only spent two minutes on the code, so perhaps I did something
horribly wrong? Could be, doubt it.

I've been asking about real-world use of object fields for quite some time
and also pushing questions and ideas towards 4D. I still don't get it.

If you're thinking about using object fields (think "subtable without a
fixed schema") ask yourself, "why?" What is it buying you? A lot of uses
I've seen are to solve problems that don't exist in a normalized table, but
do exist in a denormalized table that uses full fields. So, a lot of it
sounds to me like taking badly designed tables and redoing them into badly
design tables.

My hope for object fields was not to use them for user-facing, normalized
tables but to use them for a couple of other classes of table:

* Programmer data where you're storing what amount to manifests or
parameter blocks. So, job definitions that you pass around for execution,
logging data for forwarding to the outside world, preferences and
configuration documents of various sorts.

* Pre-built JSON blocks to pass to the outside world. In this case, 4D is
part of a pipeline where data is summarized/calculated and stored ready for
export to some other system (like a visualization platform) or ready to be
pulled by some other system (like a JavaScript that runs some kind of
rendering.) 4D told me that this is "wrong" because if I have a fixed
schema, I should use a full table. Okay, so 4D is *not* trying to build a
NoSQL/doc store system. Because if they were, then what I've described is
about as normal a use as you can come up with for those systems.

* Fine, object fields aren't meant for storing JSON blocks that have a
regular format. Well, that's probably just an opinion of some (important)
people at 4D. In practice, they're just not efficient for it as the object
takes almost exactly the amount of space required for the JSON
serialization in UTF16...which takes a ton of space, often for no purpose.

* So what are object fields for? From what 4D said in response to me, it
sounds like they're for when you have data that has an irregular format.
Hmmm. I don't have a lot of that apart from rendered JSON, logging data
(when there are a mixture of logs), or internal messages.

* Whatever problems you had with subtables, expect comparable problems with
object fields.

* I built a 1.75 million row table with an object field and did some
experiments. I couldn't find any speed advantage to using an object query
versus placing values in a field and querying there. Then again, I could
also index the object field and get the same query there, which was cool.
That seems promising. Then again, I have to use a different query command
to do this. I couldn't find any advantage to the object field per se, so
why not use a regular field/ Can't come up with a reason. I mean, I would
be willing to use a different set of commands and/or operators for some
benefit (proximity/fuzzy text searches as an obvious example, or a big
speed boost)but for no advantage? Why make the code harder.

If 4D offered some compression and/or collation options on object fields,
I'd use them for pipelining JSON. As it stands, they don't and seem
unresponsive to the suggestion.

I can absolutely see using object fields for internal messaging and prefs,
as mentioned.

Object fields for user data? While there are doubtlessly cases where this
is tempting, only  a few of those are likely to make sense. There are a few
situations where the relational model just pinches your toes. (When you
have "kind of" relations with a small number of categories.) They happen in
any big structure, but they should be uncommon. I get a bit of a tight neck
when I run into one...They're an edge case. Real, but not standard.

Using object fields for the wholesale replacement of normal fields

Re: [Off] Remote backup

[Jeffrey Kain via 4D_Tech]

Don't use 4D's code.  Read the tech note to understand the concepts, but that 
code is not good for use in the real world. 

And don't put your mirror backup code in a component either -- you'll want to 
prevent your mirror from starting a backup in the middle of an integration, and 
the only way to do this is with a semaphore. And you can't share a semaphore 
between the host and the component.

--
Jeffrey Kain
jeffrey.k...@gmail.com

> On Sep 12, 2017, at 4:01 PM, Benedict, Tom via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> there is a 4D Tech Note which includes all the code that you can need to set 
> this up. But it only supports file share or web service as the file transport 
> mechanism. Easy enough to do the LEP RoboCopy or whatever instead though.

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: [Off] Remote backup

[Benedict, Tom via 4D_Tech]

Bob Miller writes:

>[Server#1] Runs 4D Server that users are actively using.  Code in a loop on 
>the 4D Server calls
>"New Log File" every 10 minutes.  Code in the loop then calls LEP to a script 
>to transfer the last journal file to [Server#2].
>
>[Server#2] Runs 4D Server that users aren't aware of.  Code in a loop on the 
>4D Server detects
>that a complete journal file has arrived (as it will every ten minutes, my 
>uncertainty is how it knows the file transfer is
>complete) and integrates it using INTEGRATE MIRROR LOG FILE.  Code in the loop 
>then calls LEP
>to a script to transfer this journal file to [Server#3].  4D Backup runs on 
>this machine and sends its backup file to [Server#3]

>[Server#3] This is a repository, nothing is actively running on it, it could 
>be iCloud, Amazon S3, etc.

>Do I have this basically correct?

Yes.

>Then some technical questions:

>It would seem that Server#1 never runs 4D Backup, because any restoration
>would be done using Server#2.  Nevertheless, if Server#1 started up and
>somehow choked, a "normal" behavior would be to restore from backup and
>integrate the log file.  Since this behavior would be turned off, what is
>the procedure you use if Server #1 won't start?  Do you copy the mirror on
>Server #2 to Server#1?

Yes.

>How do you handle the timing, so that [Server#2] knows that the file
>transfer is complete and it is OK to integrate the log file?  Does
>[Server#1] somehow send a 'I'm Done' message to [Server#2] in your
>implementation?

I think that 4D will complain that the log file is 'busy'. I must confess I've 
never encountered this case in 10+ years of running multiple mirrors. If I 
recall correctly, our system copies the file with a 'temp' name, then when the 
copy is 'done' it changes the name. That limits the exposure considerably. It's 
pretty much an 'instant' name change, so the mirror never tries to integrate a 
'busy' file.

>What do you do if [Server#1] dies to now point users to [Server#2],
>assuming the Server application is a built application so that the server
>address and port number are built into the (built) 4D Client?

There are a number of ways to deal with this on the hostname/DNS level. We've 
never done this. Surprisingly, we never had any hardware failures when we used 
dedicated hardware and since moving to SAN/VM there have been a few drive 
failures, but they never affected operations due to the redundancy and 
'auto-move' features of VM.

BTW, there is a 4D Tech Note which includes all the code that you can need to 
set this up. But it only supports file share or web service as the file 
transport mechanism. Easy enough to do the LEP RoboCopy or whatever instead 
though.

Enterprise systems can't afford to not be mirrored. There are lots of white 
papers available which outline DR scenarios and potentials solutions for 
enterprise. Less than enterprise systems should seriously look at the very 
modest cost of deploying a 4D mirror. A few thousand dollar investment will 
provide a viable DR solution. Business interruptions due to natural disasters 
are very expensive to recover from. Being able to assure your business managers 
that the data that they depends on is safe, secure and available to deploy 
anywhere they need it is extremely compelling. Setting up a 4D Mirror is a 
simple, low cost way to satisfy that requirement.

I've advocated that 4D should build the mirroring capability into 4D Server, 
rather than providing a component. That would raise it to higher level of 
support and more systems would use mirroring. (Maybe they have? I haven't been 
following the v16 feature set closely, as I'm stuck on v13.x)

HTH,

Tom Benedict
Optum, Inc


This e-mail, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or entity
to which it is addressed. If the reader of this e-mail is not the intended
recipient or his or her authorized agent, the reader is hereby notified
that any dissemination, distribution or copying of this e-mail is
prohibited. If you have received this e-mail in error, please notify the
sender by replying to this message and delete this e-mail immediately.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

4D and Fusion Charts

[John S. Poteat via 4D_Tech]

Hi Everyone, 

I am working with Fusion Charts inside of a 4D Application and I have it 
working as expected
but there is one feature I am not able to get working get and I thought I would 
post it here in
case someone in the group has it working.

I have a dialog that contains a ListBox and WebArea. Inside of the ListBox are 
the selection
of records that makes up the data being graph in the WebArea using Fusion 
Charts.

What I would like to have happen now is if I click on an element in the Chart 
(Ex: Bar or Pie Piece)
to make a call back to 4D and have my selection of records change to show the 
records that 
correspond to the Chart element that was clicked on.

I have seen this work with Fusion Charts inside of Wakanda but I would like to 
get this working
inside of 4D.

If anyone has done something like this in 4D, I would love to hear how you did 
it. I am currently
using 4D v16R2 under Windows 10.

Thanks in advance for your help and I look forward to any responses in regards 
on my question.

Thanks,
John Poteat
jpote...@gmail.com 



**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: QUERY BY ATTRIBUTE is slow af

[Jeffrey Kain via 4D_Tech]

Yeah, so many caveats to object fields. Bite the bullet, use a related table...

--
Jeffrey Kain
jeffrey.k...@gmail.com

> On Sep 12, 2017, at 5:03 PM, David Adams via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
>> I am playing with idea to use object for values that are used only for
>> display / searching. I have structures with tables with tens of such fields
>> - mostly comments, flags or user defined fields. This would simplify
>> structure a bit. It would be nice if 4D provided some simple way how to
>> export / print values from objects (like supporting dot notation in source
>> field property.)
>> 
> 
> They do! It's called "use a related table."

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: QUERY BY ATTRIBUTE is slow af

[David Adams via 4D_Tech]

On Wed, Sep 13, 2017 at 3:22 AM, Peter Bozek via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> On Tue, Sep 12, 2017 at 7:10 PM, Herr Alexander Heintz via 4D_Tech <
> 4d_tech@lists.4d.com> wrote:
>
>
>
> I am playing with idea to use object for values that are used only for
> display / searching. I have structures with tables with tens of such fields
> - mostly comments, flags or user defined fields. This would simplify
> structure a bit. It would be nice if 4D provided some simple way how to
> export / print values from objects (like supporting dot notation in source
> field property.)
>

They do! It's called "use a related table."
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: [Off] Remote backup

[Jeffrey Kain via 4D_Tech]

Yes - you have it correct.

> It would seem that Server#1 never runs 4D Backup, because any restoration 
> would be done using Server#2.  Nevertheless, if Server#1 started up and 
> somehow choked, a "normal" behavior would be to restore from backup and 
> integrate the log file.  Since this behavior would be turned off, what is 
> the procedure you use if Server #1 won't start?  Do you copy the mirror on 
> Server #2 to Server#1?

- Yes, you'll never back up server #1 and you'll disable the automatic restore 
after crash feature.

- If server 1 crashes and you bring it up, 4D analyzes its current journal file 
and if any operations are not integrated in the data file, they are 
automatically integrated.

- If server 1 won't start (i.e. damage detected or the journal won't integrate 
for some reason), you'll need a recovery procedure.

> How do you handle the timing, so that [Server#2] knows that the file 
> transfer is complete and it is OK to integrate the log file?  Does 
> [Server#1] somehow send a 'I'm Done' message to [Server#2] in your 
> implementation?

You could send a message from #1 to #2. We just use an old-fashioned lock file. 
Write the lock file, copy the journal file, then delete the lock file. You 
could also copy the file with a .tmp extension, and when done rename .tmp to 
.journal (and your code would filter out .tmp files to prevent them from 
integrating). 

In our server, we ship a new journal file every 30 seconds. They are numbered 
sequentially, so you can sort them by file name and integrate a batch in the 
correct order.

> What do you do if [Server#1] dies to now point users to [Server#2], 
> assuming the Server application is a built application so that the server 
> address and port number are built into the (built) 4D Client?

Change the DNS entry, or manually copy the data file from #2 back to #1. We do 
the latter. It happens so infrequently that 10 minutes to recover isn't a big 
deal.

Jeff

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: QUERY BY ATTRIBUTE is slow af

[Peter Bozek via 4D_Tech]

On Tue, Sep 12, 2017 at 7:10 PM, Herr Alexander Heintz via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> Absolutely.
> I exclusively use object fields for additional data and storage of
> connected information to a specific data row, such as record and field
> history, specific item settings etc.
> All of these will NOT be queried on.
> There are cases where I use object fields for variable data structures,
> but limit this to basic 1-level objects that can be queried rapidly.
> Also bear in mind, that accessing data embedded in objects is far more
> complicated (for example for mass editing) than using a related table. It
> might sound like an intriguing concept to store, say keywords, in an object
> array, but even if these were quick to query, modifying a bunch a of
> keywords is far more complicated when they are stored in objects than just
> using an APPLY TO SELECTION for a „normal“ related table.
> Just my 2 ct, your mileage may vary
>


I am playing with idea to use object for values that are used only for
display / searching. I have structures with tables with tens of such fields
- mostly comments, flags or user defined fields. This would simplify
structure a bit. It would be nice if 4D provided some simple way how to
export / print values from objects (like supporting dot notation in source
field property.)

--

Peter Bozek
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: QUERY BY ATTRIBUTE is slow af

[Herr Alexander Heintz via 4D_Tech]

Absolutely.
I exclusively use object fields for additional data and storage of connected 
information to a specific data row, such as record and field history, specific 
item settings etc.
All of these will NOT be queried on.
There are cases where I use object fields for variable data structures, but 
limit this to basic 1-level objects that can be queried rapidly.
Also bear in mind, that accessing data embedded in objects is far more 
complicated (for example for mass editing) than using a related table. It might 
sound like an intriguing concept to store, say keywords, in an object array, 
but even if these were quick to query, modifying a bunch a of keywords is far 
more complicated when they are stored in objects than just using an APPLY TO 
SELECTION for a „normal“ related table.
Just my 2 ct, your mileage may vary


> Am 12.09.2017 um 18:58 schrieb Jeffrey Kain via 4D_Tech 
> <4d_tech@lists.4d.com>:
> 
> So disappointing.

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: QUERY BY ATTRIBUTE is slow af

[Jeffrey Kain via 4D_Tech]

So disappointing.

--
Jeffrey Kain
jeffrey.k...@gmail.com

> On Sep 12, 2017, at 12:14 PM, Herr Alexander Heintz via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
> Having worked with 4D since ages long forgotten, I would have to feel very 
> lucky and outright daring to even assume such a thing might work :-)

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

[Off] Remote backup

[bob.miller--- via 4D_Tech]

Hi Jeff,

This is a great idea, I had not thought of it.  Let me repeat it back to 
see if I have it:

[Server#1] Runs 4D Server that users are actively using.  Code in a loop 
on the 4D Server calls "New Log File" every 10 minutes.  Code in the loop 
then calls LEP to a script to transfer the last journal file to 
[Server#2].

[Server#2] Runs 4D Server that users aren't aware of.  Code in a loop on 
the 4D Server detects that a complete journal file has arrived (as it will 
every ten minutes, my uncertainty is how it knows the file transfer is 
complete) and integrates it using INTEGRATE MIRROR LOG FILE.  Code in the 
loop then calls LEP to a script to transfer this journal file to 
[Server#3].  4D Backup runs on this machine and sends its backup file to 
[Server#3]

[Server#3] This is a repository, nothing is actively running on it, it 
could be iCloud, Amazon S3, etc.


Do I have this basically correct?

Then some technical questions:

It would seem that Server#1 never runs 4D Backup, because any restoration 
would be done using Server#2.  Nevertheless, if Server#1 started up and 
somehow choked, a "normal" behavior would be to restore from backup and 
integrate the log file.  Since this behavior would be turned off, what is 
the procedure you use if Server #1 won't start?  Do you copy the mirror on 
Server #2 to Server#1?

How do you handle the timing, so that [Server#2] knows that the file 
transfer is complete and it is OK to integrate the log file?  Does 
[Server#1] somehow send a 'I'm Done' message to [Server#2] in your 
implementation?

What do you do if [Server#1] dies to now point users to [Server#2], 
assuming the Server application is a built application so that the server 
address and port number are built into the (built) 4D Client?

Many thanks!

Bob Miller
Chomerics, a division of Parker Hannifin Corporation



**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: QUERY BY ATTRIBUTE is slow af

[Herr Alexander Heintz via 4D_Tech]

Am 12.09.2017 um 17:22 schrieb Jeffrey Kain via 4D_Tech <4d_tech@lists.4d.com>:
> 
> Trying to figure out why our experience with QUERY BY ATTRIBUTE is so 
> different than the demos at the last 4D Summit.
> 
> We're running 4D Server 16.2 64-bit on Windows. The Invoices table has about 
> 9 million rows, and we added an index to the Extra field in this table which 
> is an object field.
> 
> The following query always runs sequentially - i.e. very very slowly. 
> Thinking it might be a wildcard issue with the email we also tried strings 
> with no '@', but no matter what we do the query runs sequentially.
> 
> QUERY BY ATTRIBUTE \
>  ([Invoices];\
>   [Invoices]Extra;\
>   „gift_card_line_items[].to_email";=;"desig...@sweetwater.com")

If Memory serves me right: the demos at the summit only queried on first level 
object fields:

QUERY BY ATTRIBUTE ([table]; [table]object;“name“;=;“fred“)

You, sir, are trying to get to a second level value from an array

Having worked with 4D since ages long forgotten, I would have to feel very 
lucky and outright daring to even assume such a thing might work :-)

Not helpful I know….
Try searching for a first level direct value if you have such a thing.

Cheers
Alex
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: QUERY BY ATTRIBUTE is slow af

[Jeffrey Kain via 4D_Tech]

That was my first thought too, but I can change the query string to "Ralph" and 
it still does a sequential scan of the table, returning no rows.

--
Jeffrey Kain
jeffrey.k...@gmail.com


> On Sep 12, 2017, at 11:26 AM, Spencer Hinsdale via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
> isn't there some trick to use a character code instead of at?  
> 
> and does the "beginning and end" pref possibly apply?

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: QUERY BY ATTRIBUTE is slow af

[Epperlein, Lutz (agendo) via 4D_Tech]

Maybe there is an issue with @ character in the search string ...
I've no idea how to circumvent it.

HTH
Regards
Lutz

> -Original Message-
> From: 4D_Tech [mailto:4d_tech-boun...@lists.4d.com] On Behalf Of Jeffrey Kain 
> via
> 4D_Tech
> Sent: Tuesday, September 12, 2017 5:23 PM
> To: 4d_tech@lists.4d.com
> Cc: Jeffrey Kain 
> Subject: QUERY BY ATTRIBUTE is slow af
> 
> Trying to figure out why our experience with QUERY BY ATTRIBUTE is so 
> different than
> the demos at the last 4D Summit.
> 
> We're running 4D Server 16.2 64-bit on Windows. The Invoices table has about 9
> million rows, and we added an index to the Extra field in this table which is 
> an object
> field.
> 
> The following query always runs sequentially - i.e. very very slowly. 
> Thinking it might be
> a wildcard issue with the email we also tried strings with no '@', but no 
> matter what we
> do the query runs sequentially.
> 
> QUERY BY ATTRIBUTE \
>   ([Invoices];\
>[Invoices]Extra;\
>"gift_card_line_items[].to_email";=;"desig...@sweetwater.com")
> 
> 
> What are we doing wrong here? The query returns one row, 20 minutes later.
> *
> *
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> *
> *
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: QUERY BY ATTRIBUTE is slow af

[Spencer Hinsdale via 4D_Tech]

isn't there some trick to use a character code instead of at?  

and does the "beginning and end" pref possibly apply?

> On Sep 12, 2017, at 8:22 AM, Jeffrey Kain via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> Trying to figure out why our experience with QUERY BY ATTRIBUTE is so 
> different than the demos at the last 4D Summit.
> 
> We're running 4D Server 16.2 64-bit on Windows. The Invoices table has about 
> 9 million rows, and we added an index to the Extra field in this table which 
> is an object field.
> 
> The following query always runs sequentially - i.e. very very slowly. 
> Thinking it might be a wildcard issue with the email we also tried strings 
> with no '@', but no matter what we do the query runs sequentially.
> 
> QUERY BY ATTRIBUTE \
>  ([Invoices];\
>   [Invoices]Extra;\
>   "gift_card_line_items[].to_email";=;"desig...@sweetwater.com")
> 
> 
> What are we doing wrong here? The query returns one row, 20 minutes later.
> **
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

QUERY BY ATTRIBUTE is slow af

[Jeffrey Kain via 4D_Tech]

Trying to figure out why our experience with QUERY BY ATTRIBUTE is so different 
than the demos at the last 4D Summit.

We're running 4D Server 16.2 64-bit on Windows. The Invoices table has about 9 
million rows, and we added an index to the Extra field in this table which is 
an object field.

The following query always runs sequentially - i.e. very very slowly. Thinking 
it might be a wildcard issue with the email we also tried strings with no '@', 
but no matter what we do the query runs sequentially.

QUERY BY ATTRIBUTE \
  ([Invoices];\
   [Invoices]Extra;\
   "gift_card_line_items[].to_email";=;"desig...@sweetwater.com")


What are we doing wrong here? The query returns one row, 20 minutes later.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Edit in list / subform / 4D v16

[Two Way Communications via 4D_Tech]

Ok, I tested a regular list of [Contacts], where the fields are side by side on 
1 row. Editing works here.

However, if you make the detail section bigger and put the fields vertically, 
editing is no longer possible… bug?

Regards,

Rudy Mortier
Two Way Communications bvba  -



> On 12 Sep 2017, at 13:21, Keisuke Miyako via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> to edit in list you need to click twice; that means to click an already 
> selected line with an interval longer than that of a double click. the exact 
> threshold is adjustable at the system level.
> 
> 
> 
> **
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Edit in list / subform / 4D v16

[Two Way Communications via 4D_Tech]

In the output form, fields are one above the other, so the detail section is 3 
fields high. Could that be the cause?

Regards,

Rudy Mortier
Two Way Communications bvba  -



> On 12 Sep 2017, at 13:21, Keisuke Miyako via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> to edit in list you need to click twice; that means to click an already 
> selected line with an interval longer than that of a double click. the exact 
> threshold is adjustable at the system level.
> 
> 
> 
> **
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Edit in list / subform / 4D v16

[Keisuke Miyako via 4D_Tech]

to edit in list you need to click twice; that means to click an already 
selected line with an interval longer than that of a double click. the exact 
threshold is adjustable at the system level.



**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Edit in list / subform / 4D v16

[Spencer Hinsdale via 4D_Tech]

read write([contacts])?

> On Sep 12, 2017, at 4:15 AM, Two Way Communications via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
> I thought this would be a no brainer but it’s starting to become a nuisance…
> 
> I create a subform for a table [Contacts] inside the form for the table 
> [Customers]
> The subform is set to display the [Contacts] Output form, is set to ‘Edit in 
> list’ etc.
> 
> However, when I display the list of [Contacts] inside my [Customers] input 
> form, they display fine, are selectable.
> 
> There is NO WAY however to ‘edit in list’. Whatever I do. Click, long click, 
> etc.
> And yes, the fields are set to be ‘enterable’.
> 
> This is 4D v16.1, on Mac OS Yosemite, 4D developer mode.
> 
> 
> Am I overlooking something or just getting old? ;-)
> 

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Edit in list / subform / 4D v16

[Two Way Communications via 4D_Tech]

I thought this would be a no brainer but it’s starting to become a nuisance…

I create a subform for a table [Contacts] inside the form for the table 
[Customers]
The subform is set to display the [Contacts] Output form, is set to ‘Edit in 
list’ etc.

However, when I display the list of [Contacts] inside my [Customers] input 
form, they display fine, are selectable.

There is NO WAY however to ‘edit in list’. Whatever I do. Click, long click, 
etc.
And yes, the fields are set to be ‘enterable’.

This is 4D v16.1, on Mac OS Yosemite, 4D developer mode.


Am I overlooking something or just getting old? ;-)


Regards,

Rudy Mortier
Two Way Communications bvba  -

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: OBJECT get variable type

[Keith White via 4D_Tech]

Hi

>@Piotr: can you report this in the 4D forum as feature request? So I can vote

Object filters don't do the same thing because placeholders cause the problems 
Koen already outlined.

Here is the link to the feature request for alpha variable length limit (I also 
asked for similar alpha/text array listbox column entry property):-

http://forums.4d.com/Post/EN/19424825/1/20684007#20684007

Best regards

Keith White
Synergist Express Ltd, UK.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**