Re: JSON Tools Was: Re: C-objects and memory use

[Rick Hazey via 4D_Tech]

First, let me say I wasn’t knocking 4D but answering the question Kirk Brooks 
asked: "At the risk of appearing really dense what are the specific things you 
can
do with NTK that you can't do with native 4D?”.  My answer is that JSON isn’t 
fully supported and I gave an example, so I’m not sure what the fuss is about. 

It seems to me JSON could be natively supported and the new collection type 
appears to be getting 4D closer to native tools for JSON. This is a good thing, 
IMHO.

I’ve been out of the 4D development game for a few years now but I still use 4D 
on a regular basis. I would love to see native JSON commands in 4D but I’ve 
rolled my own parser and key/value storage. This has worked quite well for me 
but if 4D provided native tools, I would switch to those in a heartbeat.

Perhaps turning this around, might be more enlightening: How many people would 
be interested in downloading a JSON parser if I uploaded it to github?



Rick Hazey 
Octet Industries, LLC 
--- 
r...@octethosting.net  
--- 
WWKD?

> On Aug 8, 2017, at 9:50 PM, David Adams via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> On Tue, Aug 8, 2017 at 5:14 PM, Keisuke Miyako via 4D_Tech <
> 4d_tech@lists.4d.com> wrote:
> 
>> it seems this view just keeps coming up...
>> 
> 
> 
>> I am sorry if the documentation or marketing material or presentation or
>> some other form of communication gave the wrong impression, but Object and
>> Array Object are not "4D's way of supporting JSON".
>> 
> 
> Thanks for saying this so clearly. This is the reality that is behind my
> feature request(s) for a set of native tools to deal with arbitrary JSON
> input/output. For now, I've got NTK but it is a feature set that any modern
> tool should have out of the box.
> **
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[Cannon Smith via 4D_Tech]

I can! :-)



--
Cannon.Smith
Synergy Farm Solutions Inc.
Hill Spring, AB Canada
403-626-3236




> On Aug 9, 2017, at 1:34 PM, David Adams via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> Sorry, I don't have a Forum link to the right request, but perhaps someone
> else can supply one?

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[David Adams via 4D_Tech]

> So many times in the past, and even still today, 4D will add a now
property to
> a form object that can only be set by using the Property List in the
Design
> environment. So no programatic way to “set” or “get” the new property.
Glaring
> example of this are new listbox properties. Then after some time and
several
> new versions of 4D, new “get” and “set” commands are added to the
language so
> you have programatic control over the property.

Good point! My favorite feature request in the past year comes from Cannon
Smith and relates to your point. Cannon asked for each form object to have
a C_OBJECT associated with it *autoamtically*. The idea is to have a little
dictionary available automatically. It's a really beautiful idea nd feels
very "4D" to me. Well, if 4D were to add this feature, it would help make
it easy to store configuration/behavioral informational object about a form
widget "in" the form widget.

And, at that point, why not extend the sytax to support native 4D
properties as well?

$font_size := MyWonderfulButton.FontSize

MyWonderfulButton.FontSize := 14

If you haven't already voted for Cannon's request, please do. (Anyone
rolling their own event cycles/MVVC/etc. stuff in native 4D alredy knows
they want this...everyone else would just wonder how they had lived without
it for so long.)

Sorry, I don't have a Forum link to the right request, but perhaps someone
else can supply one?
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[Tim Nevels via 4D_Tech]

On Aug 9, 2017, at 2:00 PM,David Adams wrote:

> Thanks for saying this so clearly. This is the reality that is behind my
> feature request(s) for a set of native tools to deal with arbitrary JSON
> input/output. For now, I've got NTK but it is a feature set that any modern
> tool should have out of the box.

When I read all these messages about how 4D has done a “half-assed” job of 
implementing native support for JSON it reminds me of object properties on 
forms. 

So many times in the past, and even still today, 4D will add a now property to 
a form object that can only be set by using the Property List in the Design 
environment. So no programatic way to “set” or “get” the new property. Glaring 
example of this are new listbox properties. Then after some time and several 
new versions of 4D, new “get” and “set” commands are added to the language so 
you have programatic control over the property. 

I think JSON support has been implemented in a similar way. Instead of “full 
support” for JSON we have “half-assed” support. Maybe that is to sever. Maybe 
it more like "3/4-assed". :) But I’m confident that in future versions of 4D 
they will add more commands to the language and complete support for JSON. For 
now we will all have to work around 4D’s limitations. 

Also consider the constant “Folder separator”. Since the beginning of time 
every 4D developer has had to implement this feature using an interprocess 
variable or a function to return either “:” or “/“. Now we don’t need to. 4D 
has provided native support for this feature. 

Tim


Tim Nevels
Innovative Solutions
785-749-3444
timnev...@mac.com


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[David Adams via 4D_Tech]

On Tue, Aug 8, 2017 at 5:14 PM, Keisuke Miyako via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> it seems this view just keeps coming up...
>


> I am sorry if the documentation or marketing material or presentation or
> some other form of communication gave the wrong impression, but Object and
> Array Object are not "4D's way of supporting JSON".
>

Thanks for saying this so clearly. This is the reality that is behind my
feature request(s) for a set of native tools to deal with arbitrary JSON
input/output. For now, I've got NTK but it is a feature set that any modern
tool should have out of the box.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[Keisuke Miyako via 4D_Tech]

it seems this view just keeps coming up...

I think it is important to understand that 4D didn't look at JSON and implement 
it as Object and Array Object, doing a really bad job at it, as it were, no, it 
happened the other way round... Object and Array Object were implemented first, 
and then JSON was chosen as a practical way to stringify these new native data 
types. as such, it should not come as a surprise at all that not every JSON can 
be converted to a 4D Object or Object Array. I am sorry if the documentation or 
marketing material or presentation or some other form of communication gave the 
wrong impression, but Object and Array Object are not "4D's way of supporting 
JSON".

> 2017/08/09 5:42、Rick Hazey via 4D_Tech <4d_tech@lists.4d.com> のメール：
>
> [ 1, 2, 3 ]
>
> 4D couldn’t parse it.




**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[Kirk Brooks via 4D_Tech]

Rick,
You have to use the JSON PARSE ARRAY command.

http://doc.4d.com/4Dv15/4D/15.4/JSON-PARSE-ARRAY.301-3274446.en.html

I wrote a wrapper for JSON strings to look at the first char and then use
the appropriate method.

This is one place where NTK is more convenient since it makes that
adjustment automatically. Which it can do since it simply returns a handle
to the newly created JSON object. 4D can't because it goes ahead and parses
the string into a 4D typed var.

On Tue, Aug 8, 2017 at 1:42 PM, Rick Hazey via 4D_Tech <4d_tech@lists.4d.com
> wrote:

> It’s been a long time since I’ve used 4D and JSON but as I recall one
> problem was the inability to parse JSON where the top level is an array
> instead of an object. IOW, if your JSON looked like this:
> [ 1, 2, 3 ]
>
> 4D couldn’t parse it.
>
-- 
Kirk Brooks
San Francisco, CA
===

*The only thing necessary for the triumph of evil is for good men to do
nothing.*

*- Edmund Burke*
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[Rick Hazey via 4D_Tech]

It’s been a long time since I’ve used 4D and JSON but as I recall one problem 
was the inability to parse JSON where the top level is an array instead of an 
object. IOW, if your JSON looked like this:

[ 1, 2, 3 ]

4D couldn’t parse it.

A pretty big deviation from the standard. On the other hand, you can test for a 
top level array, wrap it in an object, and go about your business.

I wrote a JSON parser in 4D code way back in 2011 (and did a Summit session 
about it in 2012) that has no problem handling any valid JSON. Maybe the new 
collection type will allow parsing of top level JSON arrays. Regardless, the 
collection type is a nice addition to the language.


Rick Hazey 
Octet Industries, LLC 
--- 
r...@octethosting.net  
--- 
WWKD?

> On Jul 26, 2017, at 3:13 PM, Kirk Brooks via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> ​At the risk of appearing really dense what are the specific things you can
> do with NTK that you can't do with native 4D? I use them both and I just
> don't get what you are referring to. There are differences in the way they
> provide access but I don't see how that results in a limit to what you can
> do.
> 
> And like I said currently you can use native 4D tools to parse a c-obj and
> make a change to a single key of a nested obj and the parent object is
> updated (anyone joining in here should probably read the other thread for
> the details). This is because of the object referencing we talked about in
> the other thread. NTK doesn't support that right now so you can't do it.
> Personally I find that really useful.
> 
> Can you give me an example of the kind of situation you're talking about?​

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[David Adams via 4D_Tech]

On Wed, Aug 2, 2017 at 10:51 AM, Jody Bevan via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> David:
>
> Of your four options I choose #1.  ;-)
>

I think that we all do ;-)

Short of 4D improving the language & compiler, I think the least-worst
option is rolling your own code scanner. Such an effort would also be
likely to make it easier to automate test case generation. I spent about 5
months checking these ideas out in a 4D context in the past year, and
that's where I landed.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[Jody Bevan via 4D_Tech]

David:

Of your four options I choose #1.  ;-)





Jody Bevan
ARGUS Productions Inc.
Developer
Argus Productions Inc. 




> On Aug 2, 2017, at 11:34 AM, David Adams via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> 
> Without compile-time verification of source code errors, you're left with
> these options:
> 
> * Write perfect code every time.
> 
> * Write you own code scanner. Parse your code and process whatever tests
> and checks you can use. This is very fruitful, but a ton of work. It's hard
> to see anyone but a highly motivated individual or a large team doing it.
> 
> * Write very thorough test cases to generate realistic runtime conditions
> to test your code. 4D doesn't have any real test harness available and it's
> not the simplest environment to test anyway. I've seen some good efforts
> and devoted quite a bit of time to this sort of thing myself. Again, price
> of entry is higher than nearly any 4D shop can pay. (Yes, there is a
> UnitTester component from V12 that can be converted...but it only aims to
> solve some problems and you'll still need to retrofit zillions of white box
> tests.)
> 
> * Run you code and deal with the errors when they crop up. At your
> customers.

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[David Adams via 4D_Tech]

All of the new features look great and will be most welcome. I'm on R3 now
so R4 should hopefully be an easy transition. For people on 16.1, the new
features won't appear until 17.0.

Dots will definitely make it easier to work with known element names, so
that's great.

I haven't looked at JSON Validate closely yet, but it's leveraging a draft
standard named JSON Schema:

http://json-schema.org/latest/json-schema-validation.html#rfc.section.6

Complicated. Hopefully, it will provide the kind of validation options that
we've been missing. Also hopefully, someone will break it down a bit and
write a tool to compose schemas. Ideally, from a schema you'll be able to
generate new objects generically. Meaning, pass in a schema and get out a
conformant starter object. (Without that, the feature really misses one of
the better pieces of low-hanging fruit.)

Unfortunately, JSON Validate is a *runtime* tool, so it's nothing like a
replacement for a *compiler* improvement. I don't know about you, but I
would greatly prefer to see validation performed on my code "at rest" than
at runtime. Runtime validation has it's place and I'm all for tools that
support it - but that's not ideal for anything but objects that are truly
dynamic and defined at runtime. If they're defined in the code and
accessible to the compiler should check. Sadly, this doesn't seem to be
what 4D's going to do. (Compiler improvements are rare in this respect, the
compiler doesn't test parameter lists thoroughly, doesn't recognize methods
parameters as such, etc.)

Without compile-time verification of source code errors, you're left with
these options:

* Write perfect code every time.

* Write you own code scanner. Parse your code and process whatever tests
and checks you can use. This is very fruitful, but a ton of work. It's hard
to see anyone but a highly motivated individual or a large team doing it.

* Write very thorough test cases to generate realistic runtime conditions
to test your code. 4D doesn't have any real test harness available and it's
not the simplest environment to test anyway. I've seen some good efforts
and devoted quite a bit of time to this sort of thing myself. Again, price
of entry is higher than nearly any 4D shop can pay. (Yes, there is a
UnitTester component from V12 that can be converted...but it only aims to
solve some problems and you'll still need to retrofit zillions of white box
tests.)

* Run you code and deal with the errors when they crop up. At your
customers.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[Tom Swenson via 4D_Tech]

FWIW, it sounds like the object notation in v16r4 is going to alleviate much of 
the pain in dealing with JSON objects..

JSON Validate, Collections and a NULL command!

http://blog.4d.com/en-whats-new-in-4d-v16-r4/




**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[David Adams via 4D_Tech]

> You've got a point about mixed arrays. 4D doesn't deal with them well. I
> couldn't get 4D to let me change longs[2] to an object, for instance,
> without loosing the longint values.

Yeah, that's taking me back...you can get 4D to coerce anything into text,
unless it's an object...in which case you know it's an object. But the
coercion isn't a real coercion, it can clear out values. So it's not safe.
That might have been the show stopper.

Take this the right way: I'm not looking at your code samples. I have not
dog in this fight. I checked out these features, found they were lacking in
some ways for some purposes, bitched about it on the Forums, made some
feature requests...and then got back to my life.

> ​It sounds like you are working on a project where you want to be able to
> import a data object from a json and be able to completely organize the
> database based on the data found.

Nope. I want to emulate some of the benefits of structs. 4D just doesn't
seem to see the need. I don't, can't, and won't understand that - it makes
no sense to me. Rolling my own schema+validation+object creator is a pretty
desperate cry for help on it's own. Who does that these days? No one. It's
a crazy piece of infrastructure to have to add to a language through custom
code...it's just fundamental bits of plumbing that should be there. And
then it turned out to be impossible to do satisfactorily with existing
native tools. Gave up. Fortunately, I've got a keen interest in a lot of
things, so it's not hard for me to move on to a new challenge. Squirrel!

P.S. Squirrels and Woodpeckers are two things that I really do miss in
Australia.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[David Adams via 4D_Tech]

On Wed, Jul 26, 2017 at 12:13 PM, Kirk Brooks via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> David,
> On Wed, Jul 26, 2017 at 10:14 AM, David Adams via 4D_Tech <
> 4d_tech@lists.4d.com> wrote:
>


> Can you give me an example of the kind of situation you're talking about?​
>

Incredibly jet-lagged and short on sleep, so don't expect a great answer. I
dug deep into this a few months back when I was trying to write my own
schema for validation of custom types implemented in objects. I'd done the
same in V13 using NTK's JSON commands. Couldn't get it to work in 4D
because the parsers presume specific formats. Couldn't get the types of
arrays stashed by 4D either. (4D doesn't support collections in any form in
shipping versions open to discussion here.)

As much as I've used JSON in 4D for many years, that's where I come from
with JSON. If you're in JavaScript, it's pretty normal to convert JSON
formats for various reasons. Collections, arrays of objects, objects with
arrays, straight KVPs, flattened arrays...there are a lot of common
formats. I haven't found that flexibility with the built-in commands. When
I asked on the Forums, I came away with the impression that 4D was saying
"Yes, we're incomplete in that sense, but we do this other stuff with it."
Unless I'm mis-remembering, I don't think it was much of a question that
the command set wasn't a full JSON library.

Oh wait, maybe I can offer an explanation you'll agree with. There's a set
of problems where you already know what's in the JSON. Your code "knows"
what's there and, yes, that's what there. That works fine in 4D if you're
doing 4D:4D stuff. No problem. (I'd like to generalize it to emulate
structs, but every way I mention this makes the folks at 4D seem to react
like 1) I'm honking link some kind of deranged goose and 2) A bit of pate
would go nicely right about now.)  Anyway, there's another class of
problems where you either can't know the format in advance (it's coming
from the outside word), or don't want to (because you're writing a
generalized routine to operate on JSON for some reason...I've got lots of
cases like this.) Can you do this kind of blind-but-comprehensive traversal
of a random (but valid) JSON block in 4D? Maybe. I couldn't, despite trying
and following suggestions. Perhaps it's just that it's not set up for that,
or that it's harder than it ought to be. For example, you need to know in
advance what your JSON is at the top: Object or array. Wrong input, you
blow up. I guess you could trap for that, etc. Pain.

If you're doing nothing but 4D:4D work, then there is probably nothing
missing. But that's not the world I live in. It may also be that 4D
improves and can do more. But here's the deal on how I work:

* I've got a requirement. That means that I have the requirement *now*, not
at some unknown future date, so I can only use reliable, shipping tools.
(From whatever source.) I mean, I may have a few weeks or months, but not
long enough to wait for new releases.

* I'll try out what seems to be available in 4D, possibly quite
intensively. Do the features match what I'm trying to do? If not, is there
a workaround? If the answers are "no" and "no", I'm done. I don't follow
up, I don't keep track. It's in the rear view mirror for me. So, upcoming
features are of no use or even much interest to me. I decided about 20
years ago "Only plan around released software that you have in your hands."
That's a rule of thumb I've never regretted. Don't take this to mean me
saying that 4D should do everything that I need, of course that's unlikely.
But if it doesn't, I don't have to wait.

* I do the same with any set of tools, not just 4D, by the way. Actually,
here's my more complete package for tools assessment. Be that an API,
commercial offering, piece of software, etc.

-- Try and solve a real and specific problem. These days, there are so many
options that ticking all of the big boxes may not narrow the field enough.
The details matter. So dig in deeply on a real problem and see how it goes.
This usually takes a couple of days.

-- Need help? Great! Try and get some! Contact tech support, look for
forums, SO threads, etc. It makes a *huge* difference to me how the company
responds. I'm very upfront. If I haven't given them money yet, I just say
"I haven't given you money yet, but I'm checking this out." If I think that
I'm doing something a bit unusual, I tell them so and why. Good customer
service and general communications attitude can make/break a choice.

-- Docs. Super important to me. Maybe more than some...or even more than
most. But without detailed docs, it means that I have to do way more
testing than I want just to figure out how to get things up and running.
That's a big, potentially giant time-sucking hole. *Exactly* the sort of
thing that increases the chances that a tool/product gets cut out of
consideration.
**
4D Internet Users Group (4D iNUG)
FAQ:  

Re: C-objects and memory use

[Kirk Brooks via 4D_Tech]

David,
First off I hadn't seen this when I posted on the other thread so thanks
for giving me something specific.

On Wed, Jul 26, 2017 at 10:24 AM, David Adams via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> Anyway, to your question. Check out JSON libraries in any other language
> you like, you'll see what I'm saying. You can
>
> * Freely inject any valid JSON element at any point, so arrays within
> objects, objects within arrays, whatever you need.
> * Read the same.
>
​I banged this out pretty quickly:
​

$obj:=JSON Parse("{}")

OB SET($obj;"name";"kirk")

ARRAY OBJECT($aObjs;5)

​ // would be nice to be able to specify a default value...​

$aObjs{1}:=JSON Parse("{}")
$aObjs{2}:=JSON Parse("{}")
$aObjs{3}:=JSON Parse("{}")
$aObjs{4}:=JSON Parse("{}")
$aObjs{5}:=JSON Parse("{}")

OB SET ARRAY($obj;"array1";$aObjs)

ARRAY LONGINT($aLong;3)
$aLong{1}:=1000
$aLong{2}:=2000
$aLong{3}:=3000

OB SET ARRAY($aObjs{1};"longs";$aLong)
OB SET($aObjs{1};"timeStamp";String(Current date;ISO date;Current time))

OB SET($aObjs{2};"key1";"a")

$obj2:=JSON Parse("{}")
OB SET($obj2;\
"key1";"asdfas";\
"key2";"asdfas";\
"key3";"asdfas")

OB SET($obj;"obj2-1";OB Copy($obj2))

OB SET($obj2;"key2";"I changed this one")

OB SET($aObjs{5};"obj2-2";$obj2)

4D_ALERT (JSON Stringify($obj;*))


​and the object it creates is:

​{
"name": "kirk",
"array1": [
{
"longs": [
1000,
2000,
3000
],
"timeStamp": "2017-07-26T12:29:01"
},
{
"key1": "a"
},
{},
{},
{
"obj2-2": {
"key1": "asdfas",
"key2": "I changed this one",
"key3": "asdfas"
}
}
],
"obj2-1": {
"key1": "asdfas",
"key2": "asdfas",
"key3": "asdfas"
}
}


You've got a point about mixed arrays. 4D doesn't deal with them well. I
couldn't get 4D to let me change longs[2] to an object, for instance,
without loosing the longint values.

Though it will read such a JSON - I made the change in a text editor, saved
the file and 4D parses it, correctly, to a c-obj. This is with 15.4HF3. But
it won't let me manipulate it fully. Suggests that JSON is mapped fully
internally but the tools aren't fully written.

I tried the same thing with NTK and I can, indeed, drill down into the
array and manipulate the elements independently of each other.

And I mis-spoke about not being able to change a specific key and have it
propagate up to the parent object. I'm not sure what mistake I made before
that lead me to that conclusion but I see I was wrong. Or maybe it's
because I was working in v13? Anyway, I stand corrected about NTK in v15+.

* Scan items and figure out a bit about their type. (This is a real pain in
> 4D when you use their array embedding tools.)
>
​This really gets to be program specific. If you go back to the json.org
it's defined an a text based data exchange format. So there's nothing in
the json spec about data type - it's all text. All the 'type' stuff has to
be added on to whatever the program wants to do. With the exception of
mixed arrays 4D is pretty good about managing the it's data types within a
c-obj. If I put a date value into a key I can get a date back out and I can
find out it's a data value. ​

Here's an exercise: Define a C_OBJECT that has 'rules'. These rules can
> just be in your head, but they should be something like "This element which
> is required must have a value in the range such and so. This element which
> is an array must have elements with contents in this range", etc. Figure
> out how to do it in 4D. Mostly, you can, some bits, you just can't. I
> tried. Hard. Failed.
>

​It sounds like you are working on a project where you want to be able to
import a data object from a json and be able to completely organize the
database based on the data found. That would be cool. But there's going to
have to be some negotiation beforehand about what's what I think, and the
lack of that isn't a fault on 4D's part. In the javaScript world there's
something called Simple Schema that lets you create and impose rules on
json data for things like Mongo. ​This lets you create the rules like you
mention. I can do the same thing in 4D and then have a criteria to validate
a data object against to see if I know what to do with all.


-- 
Kirk Brooks
San Francisco, CA
===

*The only thing necessary for the triumph of evil is for good men to do
nothing.*

*- Edmund Burke*
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[Keith Culotta via 4D_Tech]

Sorry, forgot to say v16R3 is required to use NEW OBJECT in the example.

Keith - CDI

> On Jul 26, 2017, at 3:07 PM, Keith Culotta via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> That's what I see here.  Same with passing the object to a new process.  So, 
> not ref-counting, or ref-counting just within a process?  
> 
> The behavior of passing an object to a method called from a form button is 
> slightly different from calling the method's commands directly in the button. 
>  It's cosmetic, the appearance of the c_object in the form does not change 
> first instance.
> 
> https://www.dropbox.com/s/v5yrxzzayqurmo5/testPtrPtr.zip?dl=0
> 
> Keith - CDI
> 
>> On Jul 25, 2017, at 3:31 PM, Peter Bozek via 4D_Tech <4d_tech@lists.4d.com> 
>> wrote:
>> 
>> On Tue, Jul 25, 2017 at 9:49 PM, Keith Culotta via 4D_Tech <
>> 4d_tech@lists.4d.com> wrote:
>> 
>> I asked 4D and was told that if object is passed to worker, worker gets
>> copy. Did not test that, but it makes sense.
>> -- 
>> Peter Bozek
> 
> **
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[Keith Culotta via 4D_Tech]

That's what I see here.  Same with passing the object to a new process.  So, 
not ref-counting, or ref-counting just within a process?  

The behavior of passing an object to a method called from a form button is 
slightly different from calling the method's commands directly in the button.  
It's cosmetic, the appearance of the c_object in the form does not change first 
instance.

https://www.dropbox.com/s/v5yrxzzayqurmo5/testPtrPtr.zip?dl=0

Keith - CDI

> On Jul 25, 2017, at 3:31 PM, Peter Bozek via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> On Tue, Jul 25, 2017 at 9:49 PM, Keith Culotta via 4D_Tech <
> 4d_tech@lists.4d.com> wrote:
> 
> I asked 4D and was told that if object is passed to worker, worker gets
> copy. Did not test that, but it makes sense.
> -- 
> Peter Bozek

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[Kirk Brooks via 4D_Tech]

David,
On Wed, Jul 26, 2017 at 10:14 AM, David Adams via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> Hey, I'm fine that 4D does whatever it wants in C_OBJECTs however they want
> to. Their program, their rules. What I was responding to was Kirk's point
> about JSON per se. If you want to use JSON as an interchange format, we
> don't have a complete set of tools natively in the 4D language. JSON is
> normally treated as a tree (well, it is a tree) that you can parse and
> navigate. In 4D, you can do this using NTK.
>
​At the risk of appearing really dense what are the specific things you can
do with NTK that you can't do with native 4D? I use them both and I just
don't get what you are referring to. There are differences in the way they
provide access but I don't see how that results in a limit to what you can
do.

And like I said currently you can use native 4D tools to parse a c-obj and
make a change to a single key of a nested obj and the parent object is
updated (anyone joining in here should probably read the other thread for
the details). This is because of the object referencing we talked about in
the other thread. NTK doesn't support that right now so you can't do it.
Personally I find that really useful.

Can you give me an example of the kind of situation you're talking about?​

-- 
Kirk Brooks
San Francisco, CA
===

*The only thing necessary for the triumph of evil is for good men to do
nothing.*

*- Edmund Burke*
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[David Adams via 4D_Tech]

On Wed, Jul 26, 2017 at 10:43 AM, Jody Bevan via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> I think 4D does put effort into documentation. The manuals are there for
each new release (that is a great start).
> The BLOGs are great pieces to get introduced to a new feature. I am
amazed at the graphics they add into the
> BLOGs (maybe I am simple). The web page too. So there is effort and money
being put in.

Yes, they definitely document all of the commands, but there's rarely
anything about trade-offs/pros-cons/best practice, real-world examples,
technical details, etc. I'm using a lot of different docs from different
places and so I'm getting a new perspective on what docs look like these
days.

I agree about the blog! I really like it. It's technical marketing material
very well done. I forget to visit often enough and wish that 4D would post
reminders on this list. Or maybe they do and I just don't notice? Anyway,
anyone who hasn't visited the blob (or done so recently), it's very good.
It's probably the best thing they've got going now of that sort. Well,
short of when Miyako says just about anything ;-)
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[Jody Bevan via 4D_Tech]

David:

You could very well be right. I don’t get to see behind the curtain to have 
even a hint.

It is the same with all aspects of a company. Jonoke became ISO 9001 certified, 
and we became better each year in communication with our users. Because the 
person at the top thought these things were a priority and put the resources in 
place. We automated a lot of communication, got the company culture moved a 
long ways to living ISO 9001. We were far from perfect, but we were better and 
better each year.

I found the programmers (on the whole) are very much against documentation. 
Trying to get them to write it was near impossible. I therefore had to 
compromise and hire some quasi technical people that would sit with the 
programmers as they needed and would write up the manual. To finish the loop 
though, I needed to get the programmer to seriously review the documentation 
and provide feedback until it was right. Some programmers did ok with this, but 
some were still against doing the work.

I think 4D does put effort into documentation. The manuals are there for each 
new release (that is a great start). The BLOGs are great pieces to get 
introduced to a new feature. I am amazed at the graphics they add into the 
BLOGs (maybe I am simple). The web page too. So there is effort and money being 
put in. 

We just want more… (sounds like my customers).


Jody Bevan
ARGUS Productions Inc.
Developer

Argus Productions Inc. 




> On Jul 26, 2017, at 11:32 AM, David Adams via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> Hey Jody,
> 
> I think to get the level of docs that we need, or that other tools have,
> you would need some dedicated tech evangelist/writers on staff and a
> commitment from management to make it a priority. Back when I was working
> on my materials for the World Tour (Southern edition), I sent an email to
> four different countries offering to rewrite some of their CALL WORKER
> documentation (gratis). I never even got a response. So, no, I don't think
> it's a management priority.
> **
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[David Adams via 4D_Tech]

Hey Jody,

I think to get the level of docs that we need, or that other tools have,
you would need some dedicated tech evangelist/writers on staff and a
commitment from management to make it a priority. Back when I was working
on my materials for the World Tour (Southern edition), I sent an email to
four different countries offering to rewrite some of their CALL WORKER
documentation (gratis). I never even got a response. So, no, I don't think
it's a management priority.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[David Adams via 4D_Tech]

Jeff's question about "why with C_OBJECT but not with BLOBs and pictures"
got me thinking a bit. Yeah, fair point. I can see in a language with no
pointer-like features why you would work things the way C_OBJECT works.
Personally, I like the explicit pointer syntax and find it helpful to be
reminded very clearly when I'm dealing with something indirectly.

So why did 4D abandon this for their
ref-counting-with-automatic-garbage-collection-but-who-really-knows-what-scope-means-now?
Haven't a clue. I can think up a bunch of different possibilities, ranging
across all positions on the Tech Ideas Continuum:

||__|__||
 Monsters! Eh?  M'eh
 Nice one   Geniuses!

You'll have to ask 4D France to explain, if they're willing.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[David Adams via 4D_Tech]

> What can you only do with NTK you can not do with 4D c-objects?
...tired right now...about 16 hours on planes. Whoever said "sleeping like
a baby" clearly hadn't met any babies.

Anyway, to your question. Check out JSON libraries in any other language
you like, you'll see what I'm saying. You can

* Freely inject any valid JSON element at any point, so arrays within
objects, objects within arrays, whatever you need.
* Read the same.
* Scan items and figure out a bit about their type. (This is a real pain in
4D when you use their array embedding tools.)

Here's an exercise: Define a C_OBJECT that has 'rules'. These rules can
just be in your head, but they should be something like "This element which
is required must have a value in the range such and so. This element which
is an array must have elements with contents in this range", etc. Figure
out how to do it in 4D. Mostly, you can, some bits, you just can't. I
tried. Hard. Failed.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[David Adams via 4D_Tech]

Hey, I'm fine that 4D does whatever it wants in C_OBJECTs however they want
to. Their program, their rules. What I was responding to was Kirk's point
about JSON per se. If you want to use JSON as an interchange format, we
don't have a complete set of tools natively in the 4D language. JSON is
normally treated as a tree (well, it is a tree) that you can parse and
navigate. In 4D, you can do this using NTK.

I was just saying that here where I live (the 21st century), languages need
the ability to read, write, and navigate arbitrary JSON. The limited
options required/supported by C_OBJECT are fine for 4D's internal purposes,
I guess, but that doesn't address all uses of JSON.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[Jody Bevan via 4D_Tech]

I understand the documentation issue - I just posted yesterday about a hole in 
the documentation. Having done a lot of documentation (still doing it), and 
being the person in charge of all operations I can understand that things fall 
between the cracks. What would be nice though is that there is a way for us to 
point out what we see as a hole in the documentation. If they want, limit it 
partners only to reduce the work load (though us partners are a whiney bunch). 
With the online documentation have a comment area at the bottom of the page - 
something like the technotes. It would be like having a few hundred editors 
going over the documentation.


> On Jul 25, 2017, at 5:07 PM, David Adams via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> I am with you on the
> lack of comprehensive documentation.

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: JSON Tools Was: Re: C-objects and memory use

[Peter Bozek via 4D_Tech]

On Wed, Jul 26, 2017 at 12:09 PM, Benedict, Tom via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> David Adams dpad...@gmail.com writes:
>
> >Well, 4D's tools for manipulating JSON are sub-standard, if your goal is
> to parse/produce JSON freely. In this respect, the XML tools (external
> libraries built into 4D) are far more standard and comprehensive.
>
> Along these lines, is there a stable and maintained library for querying
> JSON similar to XPath? I looked for one a few years ago when I was
> developing my Simple Rules Engine but found that there wasn't anything
> mature yet, so I stuck with XML/XPath.
>
> This page http://orangevolt.blogspot.com/2012/12/8-ways-to-query-
> json-structures.html#!/2012/12/8-ways-to-query-json-structures.html
> mentions a number of tools. Anyone have any experience integrating them
> with 4D?
>
>
Note that 4D objects may not be internally not implemented as  JSON data,
but (probably) like hash table - something like std::map. These as standard
does not support even simple constucts like chained keys
(anObject.subObject.subsubObject.property.)


--

Peter Bozek
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

JSON Tools Was: Re: C-objects and memory use

[Benedict, Tom via 4D_Tech]

David Adams dpad...@gmail.com writes:

>Well, 4D's tools for manipulating JSON are sub-standard, if your goal is to 
>parse/produce JSON freely. In this respect, the XML tools (external libraries 
>built into 4D) are far more standard and comprehensive.

Along these lines, is there a stable and maintained library for querying JSON 
similar to XPath? I looked for one a few years ago when I was developing my 
Simple Rules Engine but found that there wasn't anything mature yet, so I stuck 
with XML/XPath.

This page 
http://orangevolt.blogspot.com/2012/12/8-ways-to-query-json-structures.html#!/2012/12/8-ways-to-query-json-structures.html
 mentions a number of tools. Anyone have any experience integrating them with 
4D?

Tom Benedict
Optum Inc
This e-mail, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or entity
to which it is addressed. If the reader of this e-mail is not the intended
recipient or his or her authorized agent, the reader is hereby notified
that any dissemination, distribution or copying of this e-mail is
prohibited. If you have received this e-mail in error, please notify the
sender by replying to this message and delete this e-mail immediately.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[John DeSoi via 4D_Tech]

> On Jul 25, 2017, at 6:02 PM, Jeffrey Kain via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> So why not BLOBs? Why not pictures, or text variables? Why does C_OBJECT 
> flaunt the limited scoping abilities we have in 4D? The only reason I can 
> think of is because an object can contain references to other other objects, 
> and it would get really hard to explain what to do with all the parent object 
> references if a child object 10 layers deep got modified.


Also note that when objects referencing other objects have a cycle (the same 
object included in more than one path) strange performance issues popup. The 
execution time for the method below increases 40 times when a cycle is added. 
Tested 15.4 Mac.

John DeSoi, Ph.D.



C_BOOLEAN($1;$executeBug)

If (Count parameters>=1)
$executeBug:=$1
Else 
$executeBug:=True
End if 


C_LONGINT($iCol;$iRow;$rowCount;$ms;$fast;$slow;$total)
C_OBJECT($oCursor;$oAllRows;$oRow;$oName)

$colCount:=5
ARRAY OBJECT($aCol;$colCount)

For ($iCol;1;$colCount)
OB SET($aCol{$iCol};"name";"name "+String($iCol);"number";$iCol)
End for 
OB SET ARRAY($oCursor;"columns";$aCol)

$rowCount:=3000
For ($iRow;1;$rowCount)
CLEAR VARIABLE($oRow)
For ($iCol;1;$colCount)
OB SET($oRow;String($iCol);String($iCol))
End for 
OB SET($oAllRows;String($iRow);$oRow)
End for 

OB SET($oCursor;"rows";$oAllRows)
$oAllRows:=OB Get($oCursor;"rows")

$fast:=MAXLONG
$slow:=0
$total:=Milliseconds

For ($iRow;1;$rowCount)
$ms:=Milliseconds
$oRow:=OB Get($oAllRows;String($iRow))
$ms:=Milliseconds-$ms
If ($ms<$fast)
$fast:=$ms
End if 
If ($ms>$slow)
$slow:=$ms
End if 

  //If this is executed, access on $oAllRows starts taking up to 40 
times longer.
If ($executeBug & ($iRow=100))
For ($iCol;1;$colCount)
OB SET($oName;OB Get($aCol{$iCol};"name");$aCol{$iCol})
End for 
OB SET($oCursor;"names";$oName)
End if 

End for 

$total:=Milliseconds-$total

ALERT(String($total)+" ms, range "+String($fast)+"-"+String($slow))

If (Count parameters=0)
EXECUTE METHOD(Current method name;*;False)
End if 


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[Kirk Brooks via 4D_Tech]

On Tue, Jul 25, 2017 at 4:54 PM, David Adams via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> Hmmm. Well, 4D's tools for manipulating JSON are sub-standard, if your goal
> is to parse/produce JSON freely. In this respect, the XML tools (external
> libraries built into 4D) are far more standard and comprehensive.
>
​I hear you say that a lot but I don't agree. For me they have been quite
adequate. I haven't had any issue parsing or producing JSON from c-objects
or back again. Having said that it seems I'm not using c-objects is some of
the esoteric ways others appear to be. For me they have opened up great
ranges of possibilities and made interacting with web apps much easier and
faster.

And now that I have a fuller grasp of exactly how they are referenced I see
even more potential.

So, a tick on the interesting new features but no tick on basic JSON
> support. For that, you need NTK.
>
​Well one thing you can't currently do with NTK is update a single key in a
nested object. I discovered this while trying to replicate the commands
from Cannon's component in a v13 db using NTK. I'm using NTK for json quite
a lot in that database. I do like the way I can access array objects using
NTK. And being able to add items directly to an array without having to
un-pack it from the object is nice. But on the whole I find 4D's c-objects
easier to deal with on the fly, as it were. In another project where I'm
using NTK for HTTP comms for example.

What can you only do with NTK you can not do with 4D c-objects?

-- 
Kirk Brooks
San Francisco, CA
===

*The only thing necessary for the triumph of evil is for good men to do
nothing.*

*- Edmund Burke*
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[David Adams via 4D_Tech]

Kirk,

Hmmm. Well, 4D's tools for manipulating JSON are sub-standard, if your goal
is to parse/produce JSON freely. In this respect, the XML tools (external
libraries built into 4D) are far more standard and comprehensive.

The 4D-specific features of C_OBJECT (like pointer resolution and so on)
are nice and full of potential.

So, a tick on the interesting new features but no tick on basic JSON
support. For that, you need NTK. It's a pity and completely baffling to me.
But I guess I'm easily baffled or something.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[Kirk Brooks via 4D_Tech]

On Tue, Jul 25, 2017 at 4:02 PM, Jeffrey Kain via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> So why not BLOBs? Why not pictures, or text variables? Why does C_OBJECT
> flaunt the limited scoping abilities we have in 4D?

​
I think it has to do with the c-object being brought about ​from figuring
out how to handle JSON. Blobs and pictures and text vars all conform more
or less to the way 4D has conceptualized data objects pretty much forever
from my perspective. And 4D-land pretty much eschewed really embracing any
alternatives (which is why it's so much fun to use XML in 4D). Not to get
all shrinky about this but I feel that had Laurent not made the detour into
Wakanda and immersed himself in javaScript for a while our tools for JSON
would be on par with the ones for XML. The c-object is stretching 4D
conceptually and technically. From what I've heard at the Summits and such
it's changing the way 4D is built internally. So it seems likely to me this
is the future of 4D being worked out. I don't think it's a bad thing.
Frankly I think being able to manipulate JSON pretty much directly in 4D is
fabulous. Of course I've demonstrated I'm not big on the internal
architecture involved but as a user and developer I think it's pretty cool.
I'm just trying to really ground my understanding of the fundamentals so I
don't start engineering in ideas that will blow up on me in a year or two.

-- 
Kirk Brooks
San Francisco, CA
===

*The only thing necessary for the triumph of evil is for good men to do
nothing.*

*- Edmund Burke*
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[David Adams via 4D_Tech]

On Wed, Jul 26, 2017 at 9:02 AM, Jeffrey Kain via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> So why not BLOBs? Why not pictures, or text variables? Why does C_OBJECT
> flaunt the limited scoping abilities we have in 4D? The only reason I can
> think of is because an object can contain references to other other
> objects, and it would get really hard to explain what to do with all the
> parent object references if a child object 10 layers deep got modified.
>

No clue. But I figure that I've used up my quota for 2017 for complaining
about the 4D language. So, now it's up to you ;--) I am with you on the
lack of comprehensive documentation. That's pretty much inexcusable and
definitely *not* industry standard. Deep, detailed technical docs are the
baseline for anything to do with coding these days. Bar. Not. Cleared.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[Jeffrey Kain via 4D_Tech]

So why not BLOBs? Why not pictures, or text variables? Why does C_OBJECT flaunt 
the limited scoping abilities we have in 4D? The only reason I can think of is 
because an object can contain references to other other objects, and it would 
get really hard to explain what to do with all the parent object references if 
a child object 10 layers deep got modified.

--
Jeffrey Kain
jeffrey.k...@gmail.com




> On Jul 25, 2017, at 6:57 PM, David Adams via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> Is that potentially confusing? Can it lead to hard-to-figure-out bugs? Yes
> and yes. Is it a reasonable thing to do in a language? Most every language
> that I've ever looked at thinks so, so why not 4D?

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[David Adams via 4D_Tech]

One way to think about C_OBJECT is as an "alias." There's a master/single
copy somewhere and all other references are nothing but aliases/references.
Except when they're not. The OB Copy command is a way of explicitly cloning
an existing object so that now you have two distinct copies.

Is that potentially confusing? Can it lead to hard-to-figure-out bugs? Yes
and yes. Is it a reasonable thing to do in a language? Most every language
that I've ever looked at thinks so, so why not 4D? But it does take keeping
things clear in your head. The simplest approach is to figure out a few
patterns that work consistently in a way that you understand and then try
and get into more nuance. Otherwise, references can behave in ways very
like badly used globals. Very easily. (Change something here and then
unexpectedly other stuff changes. Unless it doesn't.)

As Jeff pointed out, lacking comprehensive official documentation, relying
on nuance is probably foolhardy anyway.

But, again, 4D *officially* ignores this list. (To the point of not
mentioning their excellent blog posts or tech notes, as far as I can tell),
so don't expect answers here. Post on the forums and see how it goes. But
please post back here for those of us that aren't using the forums.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[Tom DeMeo via 4D_Tech]

FWIW - This does sound like the way javascript objects work.


Tom DeMeo
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[Peter Bozek via 4D_Tech]

On Tue, Jul 25, 2017 at 9:49 PM, Keith Culotta via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> It was mentioned at the introduction of CALL WORKER that not needing
> global variables was somehow in the mix.  Is C_Object the thing that
> enables this?  In other words, have a WORKER with a $myGlobals object.  As
> long as the WORKER is alive, that object is available to all methods.
>
> Keith - CDI
>
>
I asked 4D and was told that if object is passed to worker, worker gets
copy. Did not test that, but it makes sense.

-- 

Peter Bozek
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[Jody Bevan via 4D_Tech]

I would like that objects created in a specific way could be accessed by worker 
methods. We store so many settings it is somewhat of a pain to not be able to 
use them like we do everywhere else. I have not tried this ‘feature’ and until 
it is a documented feature I likely will not try it. From all that I have read, 
and watched on workers this would defeat their stated purposes though.


Jody Bevan
ARGUS Productions Inc.
Developer
Argus Productions Inc. 




> On Jul 25, 2017, at 1:49 PM, Keith Culotta via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> It was mentioned at the introduction of CALL WORKER that not needing global 
> variables was somehow in the mix.  Is C_Object the thing that enables this? 
> In other words, have a WORKER with a $myGlobals object.  As long as the 
> WORKER is alive, that object is available to all methods.

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[Kirk Brooks via 4D_Tech]

Jeff,
On Tue, Jul 25, 2017 at 8:53 AM, Jeffrey Kain via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> You can start in, say, method A with a line like
>
>   $obj:=[Table]objectField
>
> ... and pass $obj through any number of methods as a local variable, or a
> pointer, or a process object variable. Any changes anywhere down the line
> will be made in [Table]objectField, since that's the only "real" object --
> everything else is just a reference.
>
​It would never occur to me to do that but it's an interesting case to
think about. ​But by involving an object field I would think you'd be
ultimately limited by where the record was loaded. If we make the
assignment in Process A, and then pass the object to Process B I would
expect Process B to begin working with a copy of $obj, not the actual
object. Have you tested that changes made in Process B are reflected in
[Table]objectField?

And let's say we close process A but not B. I don't think the object just
goes away.

I may have to make a db and try these things out tonight. This is pretty
interesting the more I think about it.

Regarding not being a fan of the memory heap, and recalling I'm not the
most informed computer architecture guy, what are other approaches to
managing objects in memory?

Since I do a lot of work with parsing chunks of text into arrays, for
instance, it seems like a benefit to be passing references to an object
rather than the object. I have had the impression that given:

$chunk1:=Parse_some_text

Parse_text_toArrays($chunk1;->array)

Parse_text_toArrays
​ creates a separate copy of $chunk1 whereas​

$chunk1:=Parse_some_text

Parse_text_toArrays(->$chunk1;->array)

Parse_text_toArray works with the original $chunk1 directly via
dereferencing the pointer. If I understand your point in both cases I'm
passing a reference to the object. Presumably in the first case I would
make a copy of the object as soon as I do anything to alter it. Do I have
this right?

-- 
Kirk Brooks
San Francisco, CA
===

*The only thing necessary for the triumph of evil is for good men to do
nothing.*

*- Edmund Burke*
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[Jeffrey Kain via 4D_Tech]

This is correct, and unfortunate. Different people from 4D have even called it 
a bug, but this is how it is.

You can start in, say, method A with a line like

  $obj:=[Table]objectField

... and pass $obj through any number of methods as a local variable, or a 
pointer, or a process object variable. Any changes anywhere down the line will 
be made in [Table]objectField, since that's the only "real" object -- 
everything else is just a reference.

Not a fan -- this is no better than process variables when things get complex. 
But I guess the alternative is managing your objects yourself which most people 
wouldn't like.
 
--
Jeffrey Kain
jeffrey.k...@gmail.com




> On Jul 25, 2017, at 2:29 AM, Peter Bozek via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> As I understand it, there is one 'heap' of objects, what means all objects
> are IP (where by object I mean here data structures containing data).
> Object variables are local, process or IP, but they are just references to
> this global object heap.

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[Kirk Brooks via 4D_Tech]

Hi Peter,
Great reply. You helped my understanding of the way ref-counting functions
with respect to c-objects a lot. The concept of the memory heap has only
mattered to me peripherally and so ref-counting was a similarly hazy idea.
The result of mainly dealing in 4GL languages. Anyway now I get that the
objects all exist physically in the same memory heap with the references to
them being dependent on the scope the reference is created in. Nice.

On Mon, Jul 24, 2017 at 11:29 PM, Peter Bozek via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> >  ​
> > SET($1->;"name";"Kirk")
> >
> > ​then myMethod will work the same way on the client or EOS. However I
> > suspect the time to dereference a pointer adds some time on the client
> side
> > that wouldn't be required by simply accessing the object directly. ​
> >
>
> This is something I would like to test. In fact, I never tried to pass
> pointer to EOS method and do not have clue if it will be working (or is
> allowed.)
>
​ This is something I can speak to. Josh pointed me (couldn't resist) to
using pointers with EOS methods as a way to pass a pointer to the server.
Great way to populate arrays instead of packing them in to blobs, for
instance.

​An even better tip​ is this:

On the server create a stored method that performs onerous lookups and
stores the results in IP vars - like objects or arrays. It can run as often
as necessary. The IP vars persist as long as the server is running.

Make an accessor method with the EOS property like so:

GetVar(text; pointer)

where $1 is just a label and $2 is ptr to the client var to populate. It
could look like this:

Case of

:($1="salesSummary")

$2->:=<>salesSummary

End case


You can call it using local vars on the client:
GetVar("salesSummary";->$obj). ​That's fast. And since you are accessing IP
vars on the server you don't need to worry about accessing a particular
server process - server side IP vars are available to any instance of the
'twin' process. And since you are only reading the vars you don't need to
worry about locking them on the server. Unless you are using them in some
dynamic way which isn't what I'm describing here. Works v13+.​

-- 
Kirk Brooks
San Francisco, CA
===

*The only thing necessary for the triumph of evil is for good men to do
nothing.*

*- Edmund Burke*
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: C-objects and memory use

[Peter Bozek via 4D_Tech]

On Tue, Jul 25, 2017 at 1:41 AM, Kirk Brooks via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> I want to make sure my understanding about how c-objects are handled in 4D
> memory. Mainly because it seems this understanding is cobbled together from
> a number of different sources few of which I can easily find again. So...
>
>

Kirk,

Here is how I understand objects. Not sure if it is correct, and while I
use object a bit, I cannot claim I tested all my assertions, so they can be
wrong. Anyway, it may be interesting to compare our views:


> 1) c-objects are created within the 3 4D scopes: IP, process or local.
>

As I understand it, there is one 'heap' of objects, what means all objects
are IP (where by object I mean here data structures containing data).
Object variables are local, process or IP, but they are just references to
this global object heap.

You mention that variables are something like pointers, I prefer to
consider them a references (like 'fifth object from the right') for the
following reason:
Objects are ref-counted. This means that each (global) object keeps count
how many variables (local, process and IP) is referencing this object. When
that count get to 0, object is deleted.


>
> 2) but they share some characteristics with hLists and menuRefs in that
> once created they can be referred to by different instances. For example,
> if I create an process object called "prosObj" as:
>
> { obj1: {name: Kirk, ID: 1234},
>
>obj2: {name: Joe, ID: 5678}}
>
> In this case, object heap will contain three objects, each with refcount
of 1:
{name: Kirk, ID: 1234}
{name: Joe, ID: 5678}
{obj1: reference to first object, obj2: reference to second object}


>
> I can create a local obj as:
>
> $obj:=OB GET(prosObj;"obj1")
>

you create a local (object) variable referencing first object, and refcount
of this object is increased to 2.

3) My first point of clarification is that $obj is simply referencing
> prosObj so it's not significantly increasing the amount of memory used.
>

yes, object variables are just references to objects (data structures
containing object data.)

>
> 4) Conversely if I created $obj using the copy command it would. So
>
> $obj:=OB COPY(OB GET(prosObj;obj1))
>
> would create a full copy of obj1 and take up more memory.
>

More interesting case is what happens if you copy the main object
containing references to other objects. I believe references are not
coppied, it means you will get another object with object references to
original objects (And refcounts of original objects will be increased.


>
> 5) This referencing feature also applies to objects in arrays. It's easy to
> discover this inadvertently:
>
> ARRAY OBJECT($aObjs;0)
>
> OB SET($obj;key;"new value")
>
> APPEND TO ARRAY($aObjs;$obj)
> APPEND TO ARRAY($aObjs;$obj)
> APPEND TO ARRAY($aObjs;$obj)
>
> $aObjs has 3 element each containing: {key;"new Value"}.
>

As I understand it, you have only one object  {key;"new value"} with for
variables referencing it: $obj, $aObjs{1}; $aObjs{2} and $aObjs{3}.


> If I modify $obj:
>
> OB SET($obj;key;"defined value")
>
> ​each element of $aObjs is now ​
>  {key;"new Value"}
>

you mean {key;"defined value"}?


> ​Again, I'm assuming $obj only exists in memory one time with each element
> of $aObjs referring to it.
>

Yes, this is my understanding as well.


> 6) I can clear $obj without erasing the referenced values by calling:
>
> $obj:=JSON PARSE("{}")
>

yes, this creates a new (empty) object storage and stores reference to it
into $obj, and decreases refcount to original object by one. In this
example, original object data are still references by three array elements.
When method ends and local array $aObjs is cleared, the data will be
cleared.


> 7) ​And unlike menus and hLists ​the memory occupied by $obj is released
> when the process end.
>

No, objects are released - in the sense the memory occupied by it is
cleared - when their refcount is 0. Ending method clears references from
local object variables, ending process clears references from process
variables. But if you have


C_OBJECT($obj)
OB SET($obj;key;"value")

<>ipObj:=$obj

object {key;"value"} will not be cleared when method or process ends.

​
>
> 8) Passing objects as parameters is essentially equivalent to passing
> pointers to c-obj vars EXCEPT when working with a method called with
> Execute on server - object references don't propagate when moving between
> the twin and local process. So let's say myMethod
> does this:
>

Yes, when you pass object to server process or method executed on server,
object is serialized, send to other side and there recreated.

The same seems to hold if you pass object to worker, but this I have not
tested (but if it would not be true, we would have values shared with
workers.)



>  ​
> SET($1->;"name";"Kirk")
>
> ​then myMethod will work the same way on the client or EOS. However I
> suspect the time to dereference a pointer adds some time on the client side
> that wouldn't 

Re: C-objects and memory use

[David Adams via 4D_Tech]

Kirk,

This is the sort of question that would most likely take someone from 4D
Engineering to answer. They don't participate on this list, so I guess that
you can try the Forums or open a Tech Support case (?).

Please post back here if you get any reliable information as not all of us
follow the Forums.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

C-objects and memory use

[Kirk Brooks via 4D_Tech]

I want to make sure my understanding about how c-objects are handled in 4D
memory. Mainly because it seems this understanding is cobbled together from
a number of different sources few of which I can easily find again. So...

1) c-objects are created within the 3 4D scopes: IP, process or local.

2) but they share some characteristics with hLists and menuRefs in that
once created they can be referred to by different instances. For example,
if I create an process object called "prosObj" as:

{ obj1: {name: Kirk, ID: 1234},

   obj2: {name: Joe, ID: 5678}}


I can create a local obj as:

$obj:=OB GET(prosObj;"obj1")

In this case $obj can be thought of as a pointer to prosObj.obj1 because if
I change the values in $obj these changes will also be made in ProsObj. So

OB SET($obj;"name";"Xavier")

​and then look at prosObj I'll see:

{ obj1: {name: Xavier, ID: 1234},

   obj2: {name: Joe, ID: 5678}}

​

3) My first point of clarification is that $obj is simply referencing
prosObj so it's not significantly increasing the amount of memory used.

4) Conversely if I created $obj using the copy command it would. So

$obj:=OB COPY(OB GET(prosObj;obj1))

would create a full copy of obj1 and take up more memory.

5) This referencing feature also applies to objects in arrays. It's easy to
discover this inadvertently:

ARRAY OBJECT($aObjs;0)

OB SET($obj;key;"new value")

APPEND TO ARRAY($aObjs;$obj)
APPEND TO ARRAY($aObjs;$obj)
APPEND TO ARRAY($aObjs;$obj)

$aObjs has 3 element each containing: {key;"new Value"}.

If I modify $obj:

OB SET($obj;key;"defined value")

​each element of $aObjs is now ​
 {key;"new Value"}
​:​


​Again, I'm assuming $obj only exists in memory one time with each element
of $aObjs referring to it.

6) I can clear $obj without erasing the referenced values by calling:

$obj:=JSON PARSE("{}")

7) ​And unlike menus and hLists ​the memory occupied by $obj is released
when the process end.
​

8) Passing objects as parameters is essentially equivalent to passing
pointers to c-obj vars EXCEPT when working with a method called with
Execute on server - object references don't propagate when moving between
the twin and local process. So let's say myMethod
does this:

 OB
​ ​
SET($1;"name";"Kirk")


I can execute this method locally myMethod($obj) and $obj = {"name";"Kirk"}

If I set the EOS parameter (and run it on the server) no errors are thrown
but the client side version of $obj doesn't change.

​If I change myMethod to take a pointer;

​OB
 ​
SET($1->;"name";"Kirk")

​then myMethod will work the same way on the client or EOS. However I
suspect the time to dereference a pointer adds some time on the client side
that wouldn't be required by simply accessing the object directly. ​


For these examples the memory use is trivial, of course. But as I coming to
appreciate how great c-objects are for managing a lot of data in memory I
can see a time when I might be working with significant memory allocations
where these things can really matter.


-- 
Kirk Brooks
San Francisco, CA
===

*The only thing necessary for the triumph of evil is for good men to do
nothing.*

*- Edmund Burke*
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**