Re: TEXT TO DOCUMENT With Special Characters

[Keisuke Miyako via 4D_Tech]

have you tried receiving the response with a variable declared as object?

if that doesn't work, then I guess you could receive BLOB, convert to text, and 
parse.

it is always better to receive BLOB as opposes to text, picture or object, when 
you can't be sure of what goes on on the server side.



**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: TEXT TO DOCUMENT With Special Characters

[Cannon Smith via 4D_Tech]

Hi Miyako,

This morning I figured out more exactly where the error was occurring. It isn’t 
as general as I thought and it isn’t from .json files coming from disk. 
Instead, it was only occurring when using HTTP Get to pull down a .json file 
from a website.

I’m not sure if the web server should be doing something different or what, but 
the JSON that is received by HTTP Get has (always? sometimes?) a BOM at the 
beginning of it and that is what makes JSON Parse choke.

--
Cannon.Smith
Synergy Farm Solutions Inc.
Hill Spring, AB Canada
403-626-3236




> On Aug 31, 2017, at 4:12 PM, Keisuke Miyako via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
> it's true that the commands were introduced to promote JSON, among other 
> things, but as long as you use "document to json" to restore the text, the 
> BOMs are removed anyway, so I can't see how the existence of a BOM could be 
> such a big problem.
> 
> 2017/08/31 23:25、Cannon Smith via 4D_Tech <4d_tech@lists.4d.com> のメッセージ:
> 
> The main reason I didn’t want to have a BOM character was that JSON Parse 
> chokes on such documents. I’ll file a bug for this.
> 

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: TEXT TO DOCUMENT With Special Characters

[Keisuke Miyako via 4D_Tech]

to text, obviously.

plain text file formats such as CSS or JS have no way of asserting their 
encoding, so while it is true in general that a BOM for UTF8 is unnecessary, I 
believe they do serve an important role in the context of reading or writing a 
generic plain text file.

2017/09/01 7:13、Keisuke Miyako via 4D_Tech <4d_tech@lists.4d.com>

 as long as you use "document to json"



**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: TEXT TO DOCUMENT With Special Characters

[Keisuke Miyako via 4D_Tech]

Hello,

it's true that the commands were introduced to promote JSON, among other 
things, but as long as you use "document to json" to restore the text, the BOMs 
are removed anyway, so I can't see how the existence of a BOM could be such a 
big problem.

2017/08/31 23:25、Cannon Smith via 4D_Tech <4d_tech@lists.4d.com> のメッセージ:

The main reason I didn’t want to have a BOM character was that JSON Parse 
chokes on such documents. I’ll file a bug for this.



**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4dmethod.com - Publish/Subscribe Component for 4D, David Adams

[David Adams via 4D_Tech]

> I recall that my original one had the objects
> subscribe to specific services and then the routing object would only send
> messages to objects looking for them.

Yes, that makes sense. What I showed yesterday does much the same. You want
to avoid excessive broadcasting when the recipient is going to throw away
the message as of no interest.
>
> But in this case it sends the message to any object that subscribes and if
> the object is interested it dos whatever it needs to do. If not it
ignores it.
> So I think I’ll start with your approach this time and see how that works
> first. I think it will be more than enough.

If I catch what you're thinking about correctly, you sound inclined to make
the message broker smarter. That's a solid plan, if the extra complexity
can pay for itself. I didn't talk about this aspect of the design or even
drop in the relevant slides (they make no sense out of context.)

Backing up, the MessageHub in my example does a few things:

-- Hold service definitions and subscription data.
-- Distribute messages.

So, if a service publishes 100 messages, each subscriber gets a copy of
each of those 100 messages. This is fine in a 4D app like the one I showed.
More than fine, actually. The underlying CALL FORM/CALL WORKER
implementation seems to perform *really* well. It's very, very fast. You
can stack up ridiculous numbers and messages and 4D can then chew through
them quite quickly. I mention this as someone (I think it was Tim) said
"Hey, what about speed?" So there it is: Speed is fantastic, in my
experience.

Anyway, we've got 100 messages going to a 100 subscribers, so 10,000
messages. If you were working with a distributed/network message queue, it
would (or absolutely should) add some more smarts into the message
hub/broker/center. The main tasks remain the same:

-- Hold service definitions and subscription data.
-- Distribute messages.

..but "service and subscription data" needs to be augmented. When a
subscriber expresses interest in a service, you need a filter syntax. Say
you have a service that sends out updates every hour, but you only want
them every other hour. You might have a filter syntax that lets you say:

   Send message if hour is divisible by two.

That way, you only get half as many messages along that subscription line.
Instead of distributing copies of the message to every subscriber and
having them decide "Nah, nil interest, dump" - you move some of that logic
up into the broker. To make this work and keep the tidiness of the
design...and to make sure that "the thing knows about itself", you don't
want special-case code up in the broker. Instead, you want a generalized
expression testing language so that subscribers can inject the filter rules
and the broker can interpret the rules *abstractly* without understanding
the particulars of the service, message, or subscriber. This last point is
key. If you violate that rule, then you're messing up the clean "separation
of concerns" that divides up the publisher-broker-subscriber into the most
loosely coupled, but effectively connected arrangement. It's a trap! Don't
do it.

Yeah, that's more work. In a busy network context? It makes sense - but
those systems are dealing, sometimes, with genuinely overwhelming message
flows. Millions of messages. Saving 1% of a million is a lot - and you
could save a lot more. in the case of a one-machine 4D system? I didn't
think it would be wort the trouble.

Tip: You can get much the same result with the simpler broker design by
adding more 'services'. What I called 'services' is what is often called a
'channel'. So, you could make a channel that is itself less chatty and more
specific. You could have multiple channels so that people dial into more
exactly what they want. This is less efficient than putting filter rules in
the broker because the publisher has to send multiple messages to the
message broker, but it allows you to eliminate the excess broadcasting out
of the broker. It's not as strong a design as the smarter broker, but it's
easier, less complex and probably more than enough for any 4D-local
application.

Picking up on another question I got off-line: "You didn't really talk
about pre-emptive [thread-safe] coding at all." Correctish. I said the main
thing: CALL FORM / CALL WORKER are only peripherally connected to
multi-core programming from *our* point of view. From 4D's point of view:

-- It seems to be why they decided to add these features.
-- These features help solve some problems in pre-emptive threads for which
there was no existing, sensible option.

So, it's how they got there. So what? That's a historical note and has
nothing to do with using the commands...unless you are using them for those
very specific reasons. In those cases, yeah, the interactions and rules for
pre-emptive processing matter. Laurent E. did a decent Summit presentation
on this subject that I'd recommend watching, if you're interested. Just
note that the naive/idealized 

Re: 4dmethod.com - Publish/Subscribe Component for 4D, David Adams

[truegold via 4D_Tech]

Hey Brent,

You’re welcome!

I would be very interested in "the worker watcher code soon with a little demo 
video”! especially with a little video explanation.

Please let me know when it’s available.

Appreciate,
John…



> Thanks John!  It is good to hear that people are getting some valuable input
> and discussion from these meetings :)
> 
> I have posted Davids slides on 4DMethod.  Thanks for making them available,
> David!
> https://4dmethod.files.wordpress.com/2017/08/slides.pdf
> 
> Ill also share the worker watcher code soon with a little demo video if
> people are interested.  I find that it helps give some insight as to how
> busy my workers are.
> 
> Best,
> 
> Brent
> 
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Base 64 broken in 4DIC 16.x?

[Jeffrey Kain via 4D_Tech]

Anyone else seeing Base64 encoding not working correctly in the version 16 of 
4D Internet Commands?

We haven't debugged this 100% yet, but it seems to be adding some incorrect 
padding characters to the base64 text depending on the length of the source 
message, which causes mail clients to not decode the messages correctly.

Unfortunately, UTF-8 + Base64 is the new default in 4DIC, so if you just whip 
up some code using the defaults it's not going to work.

UTF-8 and quoted-printable seems fine as a workaround

Anyone else seeing this?
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4dmethod.com - Publish/Subscribe Component for 4D, David Adams

[Brent Raymond via 4D_Tech]

Thanks John!  It is good to hear that people are getting some valuable input
and discussion from these meetings :)

I have posted Davids slides on 4DMethod.  Thanks for making them available,
David!
https://4dmethod.files.wordpress.com/2017/08/slides.pdf

Ill also share the worker watcher code soon with a little demo video if
people are interested.  I find that it helps give some insight as to how
busy my workers are.

Best,

Brent



--
Sent from: http://4d.1045681.n5.nabble.com/4D-Tech-f1376241.html
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Drag & Drop challenge - developer wanted

[Lee Hinde via 4D_Tech]

Thanks!

On Wed, Aug 30, 2017 at 11:58 PM, Daniel N. Solenthaler via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> Hi Lee,
>
> Credits to Ortwin Zillgen (http://.mettre.de/wp/ <
> http://.mettre.de/wp/>), who did some fancy Apple Script and VB
> programming for us.
>
> He is currently working on an update of the Apple Script for Sierra, since
> Apple changed something in the way attachments are loaded.
>
> Cheers,
> Daniel
>
>
>
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4dmethod.com - Publish/Subscribe Component for 4D, David Adams

[truegold via 4D_Tech]

Hey David,

Ref: "I'm sceptical that slides are worth much without a presentation but, 
well, here you go! “

True except for some of the slides were used in your presentation and they 
layout on clear diagram form some of thew points you made. Especially the one 
where you renamed the command to make it clear how they were really functioning!

So some of them will be quite useful for building our mental frameworks of 
understanding.

So thank you again!

Ref: "Oh, and the most important point about Publish/Subscribe can't be said 
too often:The think knows about itself.”

Yes!!! I built StructurePulse with that concept in mind and a Publish/Subscribe 
technique. It was written using objectTools. IN fact everything was an object 
and I only used the database for persistence.

I just started reviewing my code (although in 4D 2004) and saw your 
presentation. So I will start with this polished approach and if I need to 
change pr extend I will do so. I recall that my original one had the objects 
subscribe to specific services and then the routing object would only send 
messages to objects looking for them.

But inn this caae it sends the message to any object that subscribes and if the 
object is interested it dos whatever it needs to do. If not it ignores it. So I 
think I’ll start with your approach this time and see how that works first. I 
think it will be more than enough.

Anyway, thank you again David!

Much Appreciated,
John…

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Tip: Finding scheduling conflicts

[Kirk Brooks via 4D_Tech]

David,

On Wed, Aug 30, 2017 at 2:37 PM, David Adams via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> I'm back to tell you about a wonderful thing called "the Internet" ;-) I
> turns out that Sedgewick has a series of free courses at Coursera:
>
> https://www.coursera.org/courses?languages=en=sedgewick
>
> What I linked to yesterday comes from Algorithms 1. I checked the syllabus
> and it looks like a really thorough introductions to algorithms and data
> structures course.
>

​Coursera is sort of remarkable in this way.

My only hesitation is all the work has to be submitted in JAVA - which I
really don't know. ​Maybe someone here who does know JAVA would enroll and
a group of interested 4D devs could do a hangout or something to
participate as a group?


-- 
Kirk Brooks
San Francisco, CA
===

*The only thing necessary for the triumph of evil is for good men to do
nothing.*

*- Edmund Burke*
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: TEXT TO DOCUMENT With Special Characters

[Cannon Smith via 4D_Tech]

Thanks to everyone who responded. My first problem was using the UTF8 text 
without length constant instead of “UTF-8”. Who knew they would be different! 
Thanks for pointing that out, Justin.

The main reason I didn’t want to have a BOM character was that JSON Parse 
chokes on such documents. I’ll file a bug for this. In the mean time, I’ve 
figured out a way to move forward and make peace with BOM characters while not 
affecting legacy systems until they are upgraded. It basically amounts to 
manually removing the BOM on a few files on FTP sites for awhile, but not the 
end of the world.

Thanks again for everyone’s help.

--
Cannon.Smith
Synergy Farm Solutions Inc.
Hill Spring, AB Canada
403-626-3236




> On Aug 30, 2017, at 5:31 PM, Justin Carr via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> The first thing I notice is that you are using the old BLOB to text and TEXT 
> TO BLOB constant (UTF8 text without length) with the TEXT TO DOCUMENT 
> command. This command uses either strings such as "UTF-8" or the 
> corresponding MIBEnum ID (the documentation for CONVERT FROM TEXT lists these 
> out). The UTF8 text without length constant has a value of 6 which 
> corresponds to ISO-8859-3 using MIBEnum IDs.

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: TEXT TO DOCUMENT With Special Characters

[Arnaud de Montard via 4D_Tech]

> Le 31 août 2017 à 00:53, Cannon Smith via 4D_Tech <4d_tech@lists.4d.com> a 
> écrit :
> 
> I’m having trouble with saving Unicode characters to a file. For example, the 
> following code saves a file to disk:
> 
> C_TEXT($tText;$tFilepath)
> 
> $tText:=“©” //Copyright symbol
> $tFilepath:=System folder(Desktop)+"test.txt"
> TEXT TO DOCUMENT($tFilepath;$tText;UTF8 text without length)
> 
> When I open the file again, it only contains a “?” character.
> 
> If I use:
> 
> TEXT TO DOCUMENT($tFilepath;$tText)
> 
> it saves correctly, but then the file has a BOM character at the beginning 
> which is wreaking havoc with other systems that aren’t expecting it.
> 
> Does anyone know what I should be doing here? Thanks.

Hi Cannon, 
in addition to others… 

Before v14 introduced the "text to document/document to text" pair of commands, 
I had these wrappers:



//FS_textToDocument (path_t;text_t {;charset_t)
//writes text $2 in document $1
//creates $1 if it doesn't exist
//$3 optional charSet, default utf-8
//cf FS_documentToText
C_TEXT($1)
C_TEXT($2)
C_TEXT($3)

C_BLOB($data_x)
C_LONGINT($params_l)
C_TEXT($doc_t)
C_TEXT($text_t)
C_TEXT($charSet_t)
C_TEXT($nmc_t)

If (False)
C_TEXT(FS_textToDocument ;$1)
C_TEXT(FS_textToDocument ;$2)
C_TEXT(FS_textToDocument ;$3)
End if

$nmc_t:=Current method name
$params_l:=Count parameters
Case of
: (Not(Asserted($params_l>1;$nmc_t+" 2 params expected")))
: (Not(Asserted(Length($1)>0;$nmc_t+" $1 document path empty")))
Else
$doc_t:=$1
$text_t:=$2
$charSet_t:="utf-8"
If ($params_l>2)
$charSet_t:=$3
End if
CONVERT FROM TEXT($text_t;$charSet_t;$data_x)
BLOB TO DOCUMENT($doc_t;$data_x)
End case
//_



//FS_documentToText (path_t {;charSet_t) -> txt
//reads $1 document and returns $0 text
//$2 optional charSet, default utf-8
//cf FS_textToDocument
C_TEXT($0)
C_TEXT($1)
C_TEXT($2)

C_BLOB($data_x)
C_TEXT($doc_t)
C_LONGINT($lineEnd_l)
C_LONGINT($params_l)
C_TEXT($charSet_t)
C_TEXT($nmc_t)

If (False)
C_TEXT(FS_documentToText ;$0)
C_TEXT(FS_documentToText ;$1)
C_TEXT(FS_documentToText ;$2)
End if

$nmc_t:=Current method name
$params_l:=Count parameters
Case of
: (Not(Asserted($params_l>0;$nmc_t+" $1 document path expected")))
: (Not(Asserted(Test path name($1)=Is a document;$nmc_t+" $1 not a document 
path")))
Else
$doc_t:=$1
$charSet_t:="utf-8"
If ($params_l>1)
$charSet_t:=$2
End if
DOCUMENT TO BLOB($doc_t;$data_x)
$0:=Convert to text($data_x;$charSet_t)
End case
//_



Another caveat with TEXT TO DOCUMENT is the default line breaks management, 
platform dependent (from doc: "Document with native format (Default)...Line 
breaks are converted to the native format of the operating system: CR (carriage 
return) under OS X, CRLF (carriage return + line feed) under Windows"). 
Constant 'Document unchanged' is your friend if you don't want to be surprised… 

-- 
Arnaud de Montard 

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: Widget performance

[Piotr Chabot Stadhouders via 4D_Tech]

Hi Miyako,

Searching for some tips about widgets (subforms) and handling focus, setting 
disabled, etc. I found this link on the iNUG:
https://github.com/miyako/4d-widget-custom-picture-button/wiki

I must say I really like what you have done, and the source code is really 
helpful understanding how to implement widgets.
I would advise everybody to have a look at this widget, and others, you have 
created.

These widgets most of the time are created with 4D v14.
Would you do (some) things differently when using v16?

Thanks,

Piotr

> -Oorspronkelijk bericht-
> Van: Keisuke Miyako [mailto:keisuke.miy...@4d.com]
> Verzonden: zaterdag 26 augustus 2017 0:32
> Aan: 4D iNug Technical <4d_tech@lists.4d.com>
> Onderwerp: Re: Widget performance
> 
> there is a "duplicate by matrix" menu in the form editor, which allows you to
> quickly experiment with tens of thousands of objects.
> maybe it is the execution code setup code that is taking 2 seconds.
> 
> in my experience, a form with thousands of form objects is nothing
> uncommon.
> 
> > 2017/08/25 16:39、Piotr Chabot Stadhouders via 4D_Tech
> > <4d_tech@lists.4d.com> のメール：
> > Opening the form compiled on a quite fast Windows laptop took about 2
> seconds.
> 
> 
> 

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Drag & Drop challenge - developer wanted

[Daniel N. Solenthaler via 4D_Tech]

Hi Lee,

Credits to Ortwin Zillgen (http://.mettre.de/wp/ 
), who did some fancy Apple Script and VB 
programming for us.

He is currently working on an update of the Apple Script for Sierra, since 
Apple changed something in the way attachments are loaded.

Cheers,
Daniel

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**