Re: Simple find and replace in 4D Write Pro.

[Bernd Fröhlich via 4D_Tech]

Luc Devar:

> Now I want to simply find and replace some text in the 4DwritePro document 
> before exporting in mime html. I am looking for a simple find and replace (WR 
> find in the old 4D write). I have spent more than 2 hours trying to find a 
> way with the documentation without success. I know this should be obvious but 
> cannot find it. So I am asking for help as I cannot lose all this time on a 
> simple command.

Hi Luc,
I had a bit of a struggle with this myself. WritePro is sometimes a lot more 
compilcated than good old Write.
It boils down to this:

$lPosition:=Position("MySearchText";WP Get text($oMyWriteProObject;wk 
expressions as space))
$oRange:=WP Text 
range($oMyWriteProObject;$lPosition;$lPosition+Length("MySearchText"))
WP SET TEXT($oRange;"MyNewText";wk replace)

Hope that helps.

Greetings from Germany,
Bernd Fröhlich
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Longitude Index: B-Tree or Cluster

[John J Foster via 4D_Tech]

Hi Bernard,

My scenario was based upon the way I saw someone else implement it. And I don’t 
have miles or kilometers and don't need them for my purposes.

Yea it will either be a square or a circle not sure yet.

Does anyone know if there is an updated v17 English version of the Math4D 
component? If not I’ll have to translate the dialog labels dealing with 
longitude/latitude/distance calculations.

Appreciate,
John…


> You say "Show me all cities within 4 minutes from this longitude/latitude."
> 
> I propose to search in a square around your location, ie make a search lat +- 
> 4 miles and long +- 4 miles.
> You will get a few occurrences and then refine the search to a circle with 
> geodesic functions.
> 
> Cordialement,
> 
> Bernard Escaich

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Decode Kerberos ticket for web authentification

[Epperlein, Lutz (agendo) via 4D_Tech]

Hi,

Are there ways to decode a Kerberos ticket using 4D?

Background: We have to implement a SSO login mechanism for web users using 
Kerberos authentication.
The authorization within a web server (IIS) in front of the 4D server works so 
far. We get a http request containing a header line “Authorization: Negotiate 
…”.
After the Negotiate keyword the Kerberos ticket is following. 
The question is now, how to decode this ticket to get information about the 
logged in user, e.g. the username or groups the user belongs to.

Or is this a complete wrong approach? Should we use an extension of the IIS 
instead, which provides the wanted information in http header lines too?
The question is then, which extension?

This is with 4D Server v17.4 on Windows Server 2012 or 2016.

Regards
Lutz

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Longitude Index: B-Tree or Cluster

[John J Foster via 4D_Tech]

Hey Tim,

> Many years ago at a 4D Summit I remember...

Do you have a photographic memory? 

Good info.

Thanks,
John...

> Many years ago at a 4D Summit I remember someone asking Laurent a question of 
> how Cluster B-Tree indexes were implemented. He gave a fairly detailed 
> answer. 
> 
> Cluster B-Tree indexes are a B-Tree index, but there is only 1 node for each 
> index value. The data in the node can be several different things. 
> 
> - It could be a reference to a single record like you would have in a 
> traditional B-Tree index. It’s just a record number. 
> 
> - If the node is for 2 or more records, an array of record numbers is stored 
> in the node. He made a point of saying it is the same ARRAY LONGINT just like 
> we have in the language, so 4 bytes per record.
> 
> - At some point — he didn’t go into when it would happen — 4D switches from 
> an array of record numbers to a set of records. He said it’s the same type of 
> 4D set that we have in the language where you have 1 bit representing each 
> record that is included in the set, and all the size optimizations they do 
> for sets using compression and whatever. 
> 
> The idea is to try and balance speed of converting that list of records — 
> whether it is an array of record numbers or a set — and the amount of storage 
> needed for the node.
> 
> So when you do a typical query using a Cluster B-Tree you get a single node. 
> If the node is for 1 record is is just as fast to retrieve the record as with 
> a normal B-Tree index. If the node contains an array of 10 records numbers, 
> then the array is used to retrieve the 10 records and create a selection. 
> 
> But if the node contains a lot of record — who know exactly how many “a lot” 
> is — then a set is used to create the selection of records. And that is very 
> fast and efficient. That’s why a Cluster B-Tree index on a boolean field is 
> always super fast for 10,000 records or 1,000,000 records. Using sets makes 
> it very fast to build that selection as compared to using a normal B-Tree 
> index where you would have to walk 10,000 nodes or 1,000,000 nodes of the 
> tree to build of the selection. 
> 
> Also Cluster B-Tree indexes reduces the need to rebalance the tree as new 
> records are added to the table since it usually just adds data to an existing 
> node rather than always adding a new node for each new record.

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Anyone used Twilio SMS?

[Kirk Brooks via 4D_Tech]

I'm looking for some feedback and any code you are willing to share.

Thanks

-- 
Kirk Brooks
San Francisco, CA
==
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Print(WR_Toolbar_Classic) Error - not a Write object

[John J Foster via 4D_Tech]

Hi All,

4D v17.2 64bit Windows
Windows 10 Pro

We are using the "WR_Toolbar_Classic.4dbase” component. Reason is because it 
closely resembles the earlier 4D Write UI.

Anyway, everything works except we get an error trying to print.

"Error when executing the method “Print(WR_Toolbar_Classic)” at line number 7”

passed variable is not a Write object.

Everything else works as far as I can tell. Print Preview works. Adding a 
button and calling “WP PRINT([Letters]wpTemplate) ” works.

Since I didn’t create this component, 4D did, then there must be a fix that I 
have not been able to come up with.

Has anyone ran into this? Is there a fix?

Thanks,
John…



**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Drop User NOT working (after client disconnected using Force Quit)

[John J Foster via 4D_Tech]

Hey All,

4D v17.2 64bit Windows
Windows 10 Pro

Another issue (last month actually) that I’m not sure if there is a fix. 
Although client believes there is one! Here’s what happened:

A User (client) froze up in the application and so they excited improperly by 
quitting the 4D through the Task Manager. However they still show as being 
logged in on the server. So on 4D Server -> Users tab they highlight the User 
they want to Disconnect and hit Drop User…..it doesn’t work.

And since 4D “thinks” they were still logged in then they couldn’t reconnect as 
that user.

Tried many times and the only fix was to restart the server.

Is there something that can be done besides reminding them to not “Force Quit”, 
bringing the 4D Server down and restarting?

Anyone else experience this issues?

Thanks,
John…


**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: Anyone used Twilio SMS?

[Dennis, Neil via 4D_Tech]

> I'm looking for some feedback and any code you are willing to share.

Twilio SMS is really easy, and their docs are pretty clear.

1 - Set up and account with them
2 - Setup a URL http://api.clickatell.com/http/sendmsg?user= 
&password= &api_id= 
&to=&text=
3 - To send the text sent the URL to HTTP get

Example Code:
$lResult:=0
$tResponse:=""
$tURI:=" http://api.clickatell.com/http/sendmsg?user= 
&password= &api_id= 
&to=&text="
$lResult:= HTTP get($tURI;$tResponse)

Neil








--


Privacy Disclaimer: This message contains confidential information and is 
intended only for the named addressee. If you are not the named addressee you 
should not disseminate, distribute or copy this email. Please delete this email 
from your system and notify the sender immediately by replying to this email.  
If you are not the intended recipient you are notified that disclosing, 
copying, distributing or taking any action in reliance on the contents of this 
information is strictly prohibited.

The Alternative Investments division of UMB Fund Services provides a full range 
of services to hedge funds, funds of funds and private equity funds.  Any tax 
advice in this communication is not intended to be used, and cannot be used, by 
a client or any other person or entity for the purpose of (a) avoiding 
penalties that may be imposed on any taxpayer or (b) promoting, marketing, or 
recommending to another party any matter addressed herein.
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Drop User NOT working (after client disconnected using Force Quit)

[Randy Kaempen via 4D_Tech]

John,

> On Apr 15, 2020, at 10:28 AM, John J Foster via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
> 4D v17.2 64bit Windows
> Windows 10 Pro
> 
> Another issue (last month actually) that I’m not sure if there is a fix. 
> Although client believes there is one! Here’s what happened:
> 
> A User (client) froze up in the application and so they excited improperly by 
> quitting the 4D through the Task Manager. However they still show as being 
> logged in on the server. So on 4D Server -> Users tab they highlight the User 
> they want to Disconnect and hit Drop User…..it doesn’t work.
> 
> And since 4D “thinks” they were still logged in then they couldn’t reconnect 
> as that user.
> 
> Tried many times and the only fix was to restart the server.
> 
> Is there something that can be done besides reminding them to not “Force 
> Quit”, bringing the 4D Server down and restarting?
> 
> Anyone else experience this issues?

We have seen this numerous times on a client running a Mac server with people 
logging in from a Windows terminal server.

My understanding was that clients would automatically drop off after a certain 
period when the server could not contact them.  This does not seem to be 
working.  We also tried the drop user option and it didn’t work.  It seems to 
be a 4D problem.  If you say drop user, it needs to drop the user.

This can be a big issue when the client has a limited number of licenses and 
the users that won’t drop are preventing real users from logging back in.


Randy Kaempen
Intellex Corporation

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Drop User NOT working (after client disconnected using Force Quit)

[Randy Jaynes via 4D_Tech]

In some situations where the 'drop user’ was failing, I have been able to 
switch over to the list of processes and abort each of the user’s processes.

The user was dropped when the last process successfully aborted.

I can say that this hasn’t succeeded every time, but it has worked for me.

Randy

--
Randy Jaynes
Senior Programmer and Customer Support

http://printpoint.com • 845.687.3741 • PrintPoint, Inc • 57 Ludlow Lane • 
Palisades, NY 10964 
Please send all email contacts to supp...@printpoint.com



> On Apr 15, 2020, at 11:28 AM, John J Foster via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
> Hey All,
> 
> 4D v17.2 64bit Windows
> Windows 10 Pro
> 
> Another issue (last month actually) that I’m not sure if there is a fix. 
> Although client believes there is one! Here’s what happened:
> 
> A User (client) froze up in the application and so they excited improperly by 
> quitting the 4D through the Task Manager. However they still show as being 
> logged in on the server. So on 4D Server -> Users tab they highlight the User 
> they want to Disconnect and hit Drop User…..it doesn’t work.
> 
> And since 4D “thinks” they were still logged in then they couldn’t reconnect 
> as that user.
> 
> Tried many times and the only fix was to restart the server.
> 
> Is there something that can be done besides reminding them to not “Force 
> Quit”, bringing the 4D Server down and restarting?
> 
> Anyone else experience this issues?
> 
> Thanks,
> John…
> 
> 
> **
> 4D Internet Users Group (4D iNUG)
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Drop User NOT working (after client disconnected using Force Quit)

[Spencer Hinsdale via 4D_Tech]

Dude, 17.4 is the bomb.  The ultimate fix for our times is Sleep on SSL 
(ACI0096906).  This means you can run remote, encrypted, and shut your laptop 
for lunch and when you open again 4D client is Instant ON!

New Networking (ServerNet) works awesome.  64-bit Remote, Encrypted is FAST.  
32-bit Remote and Legacy Networking are still avail for those tardy to the 
party, who like slow.

Rumors are true!  New versions will be encrypted by default, there is a secret 
key for booting Sleeping users, and you should install 17.4 Clients with 17.4 
Server because it contains New SSL.

If you need more ammo read the Bugs.  I think 17.3 HF3 fixed the problem with 
SDI Client, Quit, still running (won't launch again), and many more...



On 4/15/20, 8:28 AM, "4D_Tech on behalf of John J Foster via 4D_Tech" 
<4d_tech-boun...@lists.4d.com on behalf of 4d_tech@lists.4d.com> wrote:

Hey All,

4D v17.2 64bit Windows


**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Drop User NOT working (after client disconnected using Force Quit)

[Jeffrey Kain via 4D_Tech]

This is what I do as well.  Drop User doesn't work at all in v17R5, but killing 
off the processes does.

The issue of ghost users sounds like our experience when we tried the 
not-so-new network layer. We don't see it very often at all with the legacy 
network.

> On Apr 15, 2020, at 12:04 PM, Randy Jaynes via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> The user was dropped when the last process successfully aborted.

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Drop User NOT working (after client disconnected using Force Quit)

[John J Foster via 4D_Tech]

Hey Spencer,

I was hoping to hear something like that. The client site has about 40-50 
clients plus a few remote and they don't like changing!

So this might be the push needed.

I’ll find and review the bug fixes.

Appreciate,
John…



> Dude, 17.4 is the bomb.  The ultimate fix for our times is Sleep on SSL 
> (ACI0096906).  This means you can run remote, encrypted, and shut your laptop 
> for lunch and when you open again 4D client is Instant ON!
> 
> New Networking (ServerNet) works awesome.  64-bit Remote, Encrypted is FAST.  
> 32-bit Remote and Legacy Networking are still avail for those tardy to the 
> party, who like slow.
> 
> Rumors are true!  New versions will be encrypted by default, there is a 
> secret key for booting Sleeping users, and you should install 17.4 Clients 
> with 17.4 Server because it contains New SSL.
> 
> If you need more ammo read the Bugs.  I think 17.3 HF3 fixed the problem with 
> SDI Client, Quit, still running (won't launch again), and many more...
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Drop User NOT working (after client disconnected using Force Quit)

[John J Foster via 4D_Tech]

Hey Randy,

Spencer has indicated that there was a bug fix which handled this.

So fingers crossed…

Thanks for sharing,
John…


> We have seen this numerous times on a client running a Mac server with people 
> logging in from a Windows terminal server.
> 
> My understanding was that clients would automatically drop off after a 
> certain period when the server could not contact them.  This does not seem to 
> be working.  We also tried the drop user option and it didn’t work.  It seems 
> to be a 4D problem.  If you say drop user, it needs to drop the user.
> 
> This can be a big issue when the client has a limited number of licenses and 
> the users that won’t drop are preventing real users from logging back in.

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Drop User NOT working (after client disconnected using Force Quit)

[John J Foster via 4D_Tech]

Hey Randy,

Yea that makes sense IU guess if 4D thinks the processes are still active.

Spencer suggested that it might have been fixed in 17.4.

Fingers crossed,
John…



> In some situations where the 'drop user’ was failing, I have been able to 
> switch over to the list of processes and abort each of the user’s processes.
> 
> The user was dropped when the last process successfully aborted.
> 
> I can say that this hasn’t succeeded every time, but it has worked for me.
> 

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Long running reports to disk intermittently disrupted and fail

[John J Foster via 4D_Tech]

Hey All,

When it rains it pours…

4D v17.2 64bit Windows
Windows 10 Pro

We have some reports that are run in a batch overnight. Have to because they 
take that long to run all of them. Anyway, since we’ve upgraded from Windows 7 
to Windows 10 there have been I’m guessing timeouts on some of these larger 
reports.

The report is running along and then we get the error: "executing the method 
“EXP_CumulativeClaims” at the lilne number 63. The .txt document cannot be 
written. File not found.”

This is line #63:
SEND PACKET($fileRef;$packet)

It varies when it quits. It’s usually saved to a network drive and IT says 
nothing is happening to disrupt the connection. And there are other reports 
where this happens as well.

The text file report size - when it works - is large:

Reserve file: 831,801kb
Claims file: 26,959kb

So these reports were created way back in 2004. The number of records has grown 
dramatically over the years. Reserves is up beyond 2.3 million records.

Is it possible that this is a client-server timeout issue? If so, how to fix?

Clients IT claims the network drive is always available. Is there a way to 
“know” for sure and test each time?

Is there a way to capture debug info which might reveal the cause? Would 
logging using 4D’s component help?

Would it be helpful and speedier to redesign the reports so they are not 
sending out each row but maybe save each row to a text variable, say 100 or 
1000 rows and then send packet?

We are open to all ideas.

Appreciate,
John…



**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Drop User NOT working (after client disconnected using Force Quit)

[Tim Nevels via 4D_Tech]

On Apr 15, 2020, at 11:05 AM, Spencer Hinsdale wrote:

> Dude, 17.4 is the bomb.  The ultimate fix for our times is Sleep on SSL 
> (ACI0096906).  This means you can run remote, encrypted, and shut your laptop 
> for lunch and when you open again 4D client is Instant ON!
> 
> New Networking (ServerNet) works awesome.  64-bit Remote, Encrypted is FAST.  
> 32-bit Remote and Legacy Networking are still avail for those tardy to the 
> party, who like slow.
> 
> Rumors are true!  New versions will be encrypted by default, there is a 
> secret key for booting Sleeping users, and you should install 17.4 Clients 
> with 17.4 Server because it contains New SSL.
> 
> If you need more ammo read the Bugs.  I think 17.3 HF3 fixed the problem with 
> SDI Client, Quit, still running (won't launch again), and many more...

Spencer is right. I’m using 4D v17.3 HF2 and it is rock solid for me as well. 
I”m not using the encrypted connection feature, but without it I’m not having 
any issues. . I can leave 4D running for days on end with no issues. That’s 
single user 4D and 4D Remote connected to 4D Server on a LAN and over a WAN.

I work on a project that connects to 4D Team Developer over the internet and 
that connection can also last for days. Maybe it helps that the WAN connection 
I’ve got a pretty good. Cable modem with 1GBbit down and 20Mbit up that 
connects to 4D Team Developer that has Google fiber 1GBbit up and 1GBbit down. 
Maybe once every 2 - 3 months I’ll have a disconnect. But to me that is very 
good for a WAN connection, compared to how it was in the past. 

I think 4D ServerNet is solid now. If you had issues with it in the past, you 
should upgrade to the current version of 4D and try it again. And if you are 
still having trouble, most likely it is because your network setup has some 
subtle, low-level and/or sporadic issues (i.e. it ain't 4D’s fault). 

You’d be surprised what old ethernet cables, old ethernet switches, old 
routers, old modems and old machines running old OS versions and mix in some 
WiFi, all put together can make you think 4D's networking is not very good. It 
works most of the time… but sometimes it falls down. It’s been that way for 
years and you think you just have to learn to live with it. That it’s normal. 
Even traditional. 

I did a complete network replacement a few years ago where I literally threw in 
the trash every single ethernet cable that I had and replaced every single one 
with CAT6 cables. I also trashed my ethernet switch/router and bought a new 
gigabit switch. Bought a new cable modem too. All machines that I use are 
running macOS 10.13 or higher on hardware that is less than 5 years old. And I 
only use Windows 10 now on PCs and in virtual environments. I use a hard wire 
EVERYWHERE to eliminate any WiFi issues in my work environment. 

My philosophy is to build a great foundation — that has no issues, not even any 
minor ones — so you can then forget about it and just use it. It has worked out 
well for me.

Tim

*
Tim Nevels
Innovative Solutions
785-749-3444
timnev...@mac.com
*

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Drop User NOT working (after client disconnected using Force Quit)

[John J Foster via 4D_Tech]

Hey Tim,

Thanks for the very detailed response. I will cull through them and see what I 
can send to the client. Their IT is very sophisticated. At one time, I was 
told, they were Microsoft’s 10th largest client? Suffice to to say they are 
well equipped - a blessing and a curse since they know everything.

Appreciate,
John...

> Spencer is right. I’m using 4D v17.3 HF2 and it is rock solid for me as well. 
> I”m not using the encrypted connection feature, but without it I’m not having 
> any issues. . I can leave 4D running for days on end with no issues. That’s 
> single user 4D and 4D Remote connected to 4D Server on a LAN and over a WAN
> …

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Long running reports to disk intermittently disrupted and fail

[John J Foster via 4D_Tech]

Hi All,

Sorry for the resend …

4D v17.2 64bit Windows
Windows 10 Pro

We have some reports that are run in a batch overnight. Have to because they 
take that long to run all of them. Anyway, since we’ve upgraded from Windows 7 
to Windows 10 there have been I’m guessing timeouts on some of these larger 
reports.

The report is running along and then we get the error: "executing the method 
“EXP_CumulativeClaims” at the lilne number 63. The .txt document cannot be 
written. File not found.”

This is line #63:
SEND PACKET($fileRef;$packet)

It varies when it quits. It’s usually saved to a network drive and IT says 
nothing is happening to disrupt the connection. And there are other reports 
where this happens as well.

The text file report size - when it works - is large:

Reserve file: 831,801kb
Claims file: 26,959kb

So these reports were created way back in 2004. The number of records has grown 
dramatically over the years. Reserves is up beyond 2.3 million records.

Is it possible that this is a client-server timeout issue? If so, how to fix?

Clients IT claims the network drive is always available. Is there a way to 
“know” for sure and test each time?

Is there a way to capture debug info which might reveal the cause? Would 
logging using 4D’s component help?

Would it be helpful and speedier to redesign the reports so they are not 
sending out each row but maybe save each row to a text variable, say 100 or 
1000 rows and then send packet?

We are open to all ideas.

Appreciate,
John…

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Long running reports to disk intermittently disrupted and fail

[James Crate via 4D_Tech]

On Apr 15, 2020, at 2:20 PM, John J Foster via 4D_Tech <4d_tech@lists.4d.com> 
wrote:
> 
> 4D v17.2 64bit Windows
> Windows 10 Pro
> 
> We have some reports that are run in a batch overnight. Have to because they 
> take that long to run all of them. Anyway, since we’ve upgraded from Windows 
> 7 to Windows 10 there have been I’m guessing timeouts on some of these larger 
> reports.
> 
> The report is running along and then we get the error: "executing the method 
> “EXP_CumulativeClaims” at the lilne number 63. The .txt document cannot be 
> written. File not found.”
> [snip]
> 
> The text file report size - when it works - is large:
> 
> Reserve file: 831,801kb
> Claims file: 26,959kb
> [snip]
> 
> Would it be helpful and speedier to redesign the reports so they are not 
> sending out each row but maybe save each row to a text variable, say 100 or 
> 1000 rows and then send packet?
> 
> We are open to all ideas.

In the time it takes to troubleshoot this, you could probably rewrite the 
report to be much faster and more reliable. 

I would start by writing the file to a local temp file, and then copying that 
file to the network drive when finished. This reduces the impact of network 
issues both for completing the operation and the network share, and makes it 
much more likely you only get a completed version on the network share for 
external consumption.

The fastest way to build the output is probably with a pre-sized blob, and 
using TEXT TO BLOB with the offset parameter. Pushing elements into a 
collection and then using collection.join() is also pretty fast, but again 
you’d have to test to see how large a text variable can actually be, and 
whether the limit was large enough both interpreted and compiled. If you don’t 
have enough RAM for a 1GB blob, smaller blobs may work with SEND PACKET.

Jim Crate

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Long running reports to disk intermittently disrupted and fail

[John J Foster via 4D_Tech]

Hey Jim,

Yea you could be right!

I do believe that creating it locally and moving to the network drive can be 
more efficient. I’m not sure how much speed different there would be.

> using TEXT TO BLOB with the offset parameter

I’m investigating this command and seeing if I can understand how to use it.

I’ve only used/seen this command in Tony’s R. BB_Decrypt/BB_Encrypt methods.

Nothing in the KB that I can find with a real world example.

Anyway, I think that’s a road I have to go down.

Appreciate,
John…


> On Apr 15, 2020, at 1:25 PM, James Crate  wrote:
> 
> In the time it takes to troubleshoot this, you could probably rewrite the 
> report to be much faster and more reliable. 
> 
> I would start by writing the file to a local temp file, and then copying that 
> file to the network drive when finished. This reduces the impact of network 
> issues both for completing the operation and the network share, and makes it 
> much more likely you only get a completed version on the network share for 
> external consumption.
> 
> The fastest way to build the output is probably with a pre-sized blob, and 
> using TEXT TO BLOB with the offset parameter. Pushing elements into a 
> collection and then using collection.join() is also pretty fast, but again 
> you’d have to test to see how large a text variable can actually be, and 
> whether the limit was large enough both interpreted and compiled. If you 
> don’t have enough RAM for a 1GB blob, smaller blobs may work with SEND PACKET.
> 
> Jim Crate

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Long running reports

[John J Foster via 4D_Tech]

Hey Mike,

> I would like to know who is reading the 800 MB report. Regardless...

No one will read them directly. These are generated export files sent to 
governing authorities who then import for their various reports. I believe 
various ISO departments for claim management statistics.

>  I went the route of creating an array of text entries, then for loop the 
> array to blob, then blob to document. 

I have been thinking along similar lines today. Just wondering if keeping 2.4 
million rows in a text array in memory will be an issue. The text file is 831mb 
so if 4D can mange it that’s worth a try. Likely faster as one of the reports 
is taking 3-4 hours. And there are dozens of reports they need to run at the 
end of the month. That’s why it’s an overnight affair. (Although we could 
distribute the effort over several computers if we need to.)

Thanks for the code snippets!!!

John…


> On Apr 15, 2020, at 12:21 PM, Mike Beatty  > wrote:
> 
> John-
> 
> Re:
> 
>> Hi All,
>> 
>> Sorry for the resend …
>> 
>> 4D v17.2 64bit Windows
>> Windows 10 Pro
>> 
>> We have some reports that are run in a batch overnight. Have to because they 
>> take that long to run all of them. Anyway, since we’ve upgraded from Windows 
>> 7 to Windows 10 there have been I’m guessing timeouts on some of these 
>> larger reports.
>> 
>> The report is running along and then we get the error: "executing the method 
>> “EXP_CumulativeClaims” at the lilne number 63. The .txt document cannot be 
>> written. File not found.”
>> 
>> This is line #63:
>> SEND PACKET($fileRef;$packet)
>> 
>> It varies when it quits. It’s usually saved to a network drive and IT says 
>> nothing is happening to disrupt the connection. And there are other reports 
>> where this happens as well.
>> 
>> The text file report size - when it works - is large:
>> 
>> Reserve file: 831,801kb
>> Claims file: 26,959kb
>> 
>> So these reports were created way back in 2004. The number of records has 
>> grown dramatically over the years. Reserves is up beyond 2.3 million records.
>> 
>> Is it possible that this is a client-server timeout issue? If so, how to fix?
>> 
>> Clients IT claims the network drive is always available. Is there a way to 
>> “know” for sure and test each time?
>> 
>> Is there a way to capture debug info which might reveal the cause? Would 
>> logging using 4D’s component help?
>> 
>> Would it be helpful and speedier to redesign the reports so they are not 
>> sending out each row but maybe save each row to a text variable, say 100 or 
>> 1000 rows and then send packet?
>> 
>> We are open to all ideas.
>> 
>> Appreciate,
>> John…
> 
> I would like to know who is reading the 800 MB report. Regardless, I went the 
> route of creating an array of text entries, then for loop the array to blob, 
> then blob to document. Looks like this:
> 
> 
> ARRAY TEXT($atAccountID;0)
> ARRAY TEXT($atAccountRef;0)
> ARRAY DATE($adCloseDate;0)
> ARRAY REAL($arAccountBalance;0)
> 
> SELECTION TO 
> ARRAY([Account]AccountID;$atAccountID;[Account]AccountRef;$atAccountRef;[Account]StatusDate;$adCloseDate;[Account]OutsBalance;$arAccountBalance)
> 
> ARRAY TEXT($atExportArray;0)
> 
> For ($i;1;Size of array($atAccountID))
> $vtText:=$atAccountID{$i}+<>Pipe+$atAccountRef{$i}+<>Pipe+"WD"+<>Pipe+fDateString
>  
> ($adCloseDate{$i};"MMDD")+<>Pipe+String($arAccountBalance{$i};"###.00")+<>EOL
> 
> APPEND TO ARRAY($atExportArray;$vtText)
> 
> End for 
> 
> C_BLOB($vbExportBlob)
> SET BLOB SIZE($vbExportBlob;0)
> 
> For ($i;1;Size of array($atExportArray))
> 
> TEXT TO BLOB($atExportArray{$i};$vbExportBlob;UTF8 text without length;*)
> 
> End for 
> 
> $vtFile:="myfilename.txt"  //standrad file format for client
> $vtFile:=fReturnPath ($vtFile)
> 
> 
> BLOB TO DOCUMENT($vtFile;$vbExportBlob)
> 
> HTH.
> 
> Mike B


**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Long running reports to disk intermittently disrupted and fail

[Neil Dennis via 4D_Tech]

> The fastest way to build the output is probably with a pre-sized blob, and 
> using TEXT TO BLOB with the offset parameter. Pushing elements into a 
> collection and then using collection.join() is also pretty fast, but again 
> you’d have to test to see how large a text variable can actually be, and 
> whether the limit was large enough both interpreted and compiled. If you 
> don’t have enough RAM for a 1GB blob, smaller blobs may work with SEND PACKET.

I ran some speed tests using 4D v18 on Windows 10. The results were 
interesting… in times past it was much faster to run a report using an internal 
buffer of some sort such as a text or blob variable. My resent tests showed it 
was faster to use SEND PACKET without buffering anything.

Again I was testing with 4D v18 and results may vary.

I’m assuming you are using unicode on your app? I think you can still do that, 
but I’m not sure. If you are not in unicode you should switch.

With unicode on a text can be very large. For imports I just use Document to 
Text. When exporting, large text vars can slow down as the resizing of them 
becomes slow. You could try filling a blob, but like I said SEND PACKT seemed 
to me to be the fastest option.

As was mentioned be sure to build your document local in the temp folder then 
move it to its home on the network. Writing across the network to create a file 
is slow and prone to error.

Neil


--
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Long running reports to disk intermittently disrupted and fail

[John J Foster via 4D_Tech]

Hey Neil,

That’s interesting. This client, although in 4D v17.2 (yes unicode), hates 
upgrading. So I will have to code in v17 efficiency.

I think I’d be find with the slowness if I can figure out why the routine 
either stalls or timeout or suddenly cannot find the disk.

I do think writing the file locally and then moving it makes a lot of sense. 
Right now they choose the path to save files to but it would b easy enough to 
create it locally move doc into the path.

Thanks for the ideas!

John…


> On Apr 15, 2020, at 1:48 PM, Neil Dennis  wrote:
> 
>> The fastest way to build the output is probably with a pre-sized blob, and 
>> using TEXT TO BLOB with the offset parameter. Pushing elements into a 
>> collection and then using collection.join() is also pretty fast, but again 
>> you’d have to test to see how large a text variable can actually be, and 
>> whether the limit was large enough both interpreted and compiled. If you 
>> don’t have enough RAM for a 1GB blob, smaller blobs may work with SEND 
>> PACKET.
> 
> I ran some speed tests using 4D v18 on Windows 10. The results were 
> interesting… in times past it was much faster to run a report using an 
> internal buffer of some sort such as a text or blob variable. My resent tests 
> showed it was faster to use SEND PACKET without buffering anything.
> 
> Again I was testing with 4D v18 and results may vary.
> 
> I’m assuming you are using unicode on your app? I think you can still do 
> that, but I’m not sure. If you are not in unicode you should switch.
> 
> With unicode on a text can be very large. For imports I just use Document to 
> Text. When exporting, large text vars can slow down as the resizing of them 
> becomes slow. You could try filling a blob, but like I said SEND PACKT seemed 
> to me to be the fastest option.
> 
> As was mentioned be sure to build your document local in the temp folder then 
> move it to its home on the network. Writing across the network to create a 
> file is slow and prone to error.
> 
> Neil

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Word count in 4D write pro

[Eric Naujock via 4D_Tech]

Is there a simple way to get a accurate word count for a 4d Write pro object?  
I am trying to be Abe to check if a document is long enough to be plausibly 
done. In the past with 4D write I could convert the document to raw text and 
use STR_Count_Words to be able to get a word count. But with the new way 4D 
Write Pro extracts text out of the Write pro object I am finding that I do not 
get a real word count. Has anybody else had something that can generate an 
accurate word count for a given 4D Write Pro Object
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Word count in 4D write pro

[Randy Kaempen via 4D_Tech]

Eric,

> On Apr 15, 2020, at 4:48 PM, Eric Naujock via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> Is there a simple way to get a accurate word count for a 4d Write pro object? 
>  I am trying to be Abe to check if a document is long enough to be plausibly 
> done. In the past with 4D write I could convert the document to raw text and 
> use STR_Count_Words to be able to get a word count. But with the new way 4D 
> Write Pro extracts text out of the Write pro object I am finding that I do 
> not get a real word count. Has anybody else had something that can generate 
> an accurate word count for a given 4D Write Pro Object

What is wrong with the plain text you get out of 4D Write that gives a wrong 
word count?


Randy Kaempen
Intellex Corporation

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Print(WR_Toolbar_Classic) Error - not a Write object

[John J Foster via 4D_Tech]

Hi All,

The offending command is:

WP PRINT(WP_WriteProArea)

It appears that the component is expecting “WP_WriteProArea” to be an object. 

I don’t recall the ins and outs off passing params from the host to the 
component. But I do know that the host and component have different scopes. So 
you cannot create the same variable in the host and expect it to be seen by the 
component.

So how does the component get the “WP_WriteProArea” variable?

Anyone have any idea?

Thanks,
John…



> 4D v17.2 64bit Windows
> Windows 10 Pro
> 
> We are using the "WR_Toolbar_Classic.4dbase” component. Reason is because it 
> closely resembles the earlier 4D Write UI.
> 
> Anyway, everything works except we get an error trying to print.
> 
> "Error when executing the method “Print(WR_Toolbar_Classic)” at line number 7”
> 
> passed variable is not a Write object.
> 
> Everything else works as far as I can tell. Print Preview works. Adding a 
> button and calling “WP PRINT([Letters]wpTemplate) ” works.
> 
> Since I didn’t create this component, 4D did, then there must be a fix that I 
> have not been able to come up with.
> 
> Has anyone ran into this? Is there a fix?
> 
> Thanks,
> John…
> 
> 
> 

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Word count in 4D write pro

[Eric Naujock via 4D_Tech]

When I do an export of the 4D Write Pro document into. Text object it always 
exports as a mime type or an HTML with nonbreaking spaces document with other 
elements. I can take a 5 page 4D Write Pro document Export it a plain text and 
using the STR_Count_Words function and it will give me a count of 19 words. 
Where when it was in 4D Write it would return 19,365 words. There seems to be a 
slight error in something. Both of the below will fail if the to get e a usable 
word count.



WP EXPORT VARIABLE(WrProObject;$exportText;wk web page html 4D;wk normal)   
ProtocolWC:=STR_Count_Words ($exportText)

WP EXPORT VARIABLE(WrProObject;$exportText;wk mime html;wk normal)  

ProtocolWC:=STR_Count_Words ($exportText)

If there is a way to extract a raw plain text version of the text from the 4D 
Write pro object I would love to find out how.

> On Apr 15, 2020, at 5:50 PM, Randy Kaempen  wrote:
> 
> Eric,
> 
>> On Apr 15, 2020, at 4:48 PM, Eric Naujock via 4D_Tech <4d_tech@lists.4d.com> 
>> wrote:
>> 
>> Is there a simple way to get a accurate word count for a 4d Write pro 
>> object?  I am trying to be Abe to check if a document is long enough to be 
>> plausibly done. In the past with 4D write I could convert the document to 
>> raw text and use STR_Count_Words to be able to get a word count. But with 
>> the new way 4D Write Pro extracts text out of the Write pro object I am 
>> finding that I do not get a real word count. Has anybody else had something 
>> that can generate an accurate word count for a given 4D Write Pro Object
> 
> What is wrong with the plain text you get out of 4D Write that gives a wrong 
> word count?
> 
> 
> Randy Kaempen
> Intellex Corporation
> 

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Word count in 4D write pro

[Randy Kaempen via 4D_Tech]

Eric,

> On Apr 15, 2020, at 5:00 PM, Eric Naujock  wrote:
> 
> When I do an export of the 4D Write Pro document into. Text object it always 
> exports as a mime type or an HTML with nonbreaking spaces document with other 
> elements. I can take a 5 page 4D Write Pro document Export it a plain text 
> and using the STR_Count_Words function and it will give me a count of 19 
> words. Where when it was in 4D Write it would return 19,365 words. There 
> seems to be a slight error in something. Both of the below will fail if the 
> to get e a usable word count.
> 
> 
> 
> WP EXPORT VARIABLE(WrProObject;$exportText;wk web page html 4D;wk normal) 
> ProtocolWC:=STR_Count_Words ($exportText)
> 
> WP EXPORT VARIABLE(WrProObject;$exportText;wk mime html;wk normal)
> 
> ProtocolWC:=STR_Count_Words ($exportText)
> 
> If there is a way to extract a raw plain text version of the text from the 4D 
> Write pro object I would love to find out how.

A lot of the former 4D Write commands are now the styled text commands.

$Text:=ST Get plain text($FieldPtr->)


Randy Kaempen
Intellex Corporation

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Long running reports to disk intermittently disrupted and fail

[James Crate via 4D_Tech]

On Apr 15, 2020, at 4:35 PM, John J Foster via 4D_Tech <4d_tech@lists.4d.com> 
wrote:
> 
> I do believe that creating it locally and moving to the network drive can be 
> more efficient. I’m not sure how much speed different there would be.
> 
>> using TEXT TO BLOB with the offset parameter
> 
> I’m investigating this command and seeing if I can understand how to use it.

If your machine is on an SSD, there will probably be little difference between 
writing each row with SEND PACKET or buffering using a BLOB. If you want to 
buffer using a blob, the general strategy is like this:

// Calculate rough target size of blob plus a little extra. If 2.3M rows was 
830MB, you are at roughly 2700 rows per MB, so maybe calculate 1MB per 2500 rows
$blobSize:=(number of records / 2500)*(1024*1024)
SET BLOB SIZE($buffer;$blobSize)
$offset:=0
For ($i;1;number of records)
  // build line into $line
  // make sure we have enough space in the blob
  If (Blob size($buffer)<($offset+Length($line))
$blobSize:=$blobSize+(Length($line)*2000)// increase by enough that we 
don’t have to do it often
SET BLOB SIZE($buffer;$blobSize)
  End if
  TEXT TO BLOB($line;$buffer;UTF8 text without length;$offset)
End for
// Truncate blob to correct length
SET BLOB SIZE($buffer;$offset) // blobs are zero-indexed


Jim Crate

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Long running reports to disk intermittently disrupted and fail

[John J Foster via 4D_Tech]

Hey Jim,

> If your machine is on an SSD…

That makes sense to my very simple mind! 

I don’t think in all my years of 4D dating back to beta Silver Surfer days I 
have ever needed a routine where buffering using a BLOB was required. That’s 
amazing! Anyway...

Thanks again for your advice and for the code!

Makes it so much easier to think it through (see and build upon).

Appreciate,
John…


> On Apr 15, 2020, at 3:19 PM, James Crate  wrote:
> 
> On Apr 15, 2020, at 4:35 PM, John J Foster via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
>> 
>> I do believe that creating it locally and moving to the network drive can be 
>> more efficient. I’m not sure how much speed different there would be.
>> 
>>> using TEXT TO BLOB with the offset parameter
>> 
>> I’m investigating this command and seeing if I can understand how to use it.
> 
> If your machine is on an SSD, there will probably be little difference 
> between writing each row with SEND PACKET or buffering using a BLOB. If you 
> want to buffer using a blob, the general strategy is like this:
> 
> // Calculate rough target size of blob plus a little extra. If 2.3M rows was 
> 830MB, you are at roughly 2700 rows per MB, so maybe calculate 1MB per 2500 
> rows
> $blobSize:=(number of records / 2500)*(1024*1024)
> SET BLOB SIZE($buffer;$blobSize)
> $offset:=0
> For ($i;1;number of records)
>  // build line into $line
>  // make sure we have enough space in the blob
>  If (Blob size($buffer)<($offset+Length($line))
>$blobSize:=$blobSize+(Length($line)*2000)// increase by enough that we 
> don’t have to do it often
>SET BLOB SIZE($buffer;$blobSize)
>  End if
>  TEXT TO BLOB($line;$buffer;UTF8 text without length;$offset)
> End for
> // Truncate blob to correct length
> SET BLOB SIZE($buffer;$offset) // blobs are zero-indexed
> 
> 
> Jim Crate
> 

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Long running reports

[Chip Scheide via 4D_Tech]

John,
I've played with large files (reading) and 4D - and around 500 megs it starts 
to get wonky.

I would:
use an SSD
create the (local) disk file (blob to doc with an empty blob.
repeat 
build up your text to some size (100 megs?) then append it to the disk 
file
until no more text
move file to network

should give you maximum stability


> 
> I have been thinking along similar lines today. Just wondering if 
> keeping 2.4 million rows in a text array in memory will be an issue. 
> The text file is 831mb so if 4D can mange it that’s worth a try. 
> Likely faster as one of the reports is taking 3-4 hours. And there 
> are dozens of reports they need to run at the end of the month. 
> That’s why it’s an overnight affair. (Although we could distribute 
> the effort over several computers if we need to.)

Hell is other people 
 Jean-Paul Sartre
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**