Re: poor performance LISTBOX INSERT COLUMN

[Arnaud de Montard via 4D_Tech]

> Le 5 déc. 2017 à 14:31, Piotr Chabot Stadhouders via 4D_Tech 
> <4d_tech@lists.4d.com> a écrit :
> 
> Hi Arnaud,
> 
> Thanks for your thorough test.
> [...]
> In my tests I also saw this happening. Duplicating each listbox in the form 
> editor with 20 columns took longer and longer.
> I didn't know / expected this to be faster in runtime.

still on it… 

a/ create a form, add one array listbox of 50 columns --> in the MSC the 4db 
size is 320Kb
b/ duplicate this listbox in order to obtain 10 LB of 50 columns each --> in 
the MSC the 4db size now is 1,43Mb 

So each added listbox "cost" in that form is ~120Kb ((1430-320)/9). That may be 
another good reason to prefer duplicate at runtime. I think too that the more 
complex is a form, the more chances I have to get it corrupted one day. 

-- 
Arnaud de Montard 




**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: poor performance LISTBOX INSERT COLUMN

[Piotr Chabot Stadhouders via 4D_Tech]

Hi Tim,

Yes, I have opened a tech support case.

Gr,

Piotr

> -Oorspronkelijk bericht-
> Van: Timothy Penner [mailto:tpen...@4d.com]
> Verzonden: woensdag 6 december 2017 2:24
> Aan: 4D iNug Technical <4d_tech@lists.4d.com>
> Onderwerp: RE: poor performance LISTBOX INSERT COLUMN
> 
> > My only hope is that 4D will investigate the command.
> 
> If you haven't already done so, please open a tech support case on taow.4d.com
> or file a bug report on forums.4d.fr so that someone investigates this for 
> you.
> 
> -Tim
> 

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: poor performance LISTBOX INSERT COLUMN

[Timothy Penner via 4D_Tech]

> My only hope is that 4D will investigate the command.

If you haven't already done so, please open a tech support case on taow.4d.com 
or file a bug report on forums.4d.fr so that someone investigates this for you.

-Tim


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: poor performance LISTBOX INSERT COLUMN

[Piotr Chabot Stadhouders via 4D_Tech]

Hi Chip,

I understand what you are saying.
I did a quick test between append (position = column count + 1) and insert 
(position = 1)
I don't have the feeling it makes a lot of difference.
And remember, we are not talking about millions of elements. Maybe then it 
would make a difference.
We are talking about inserting 400 empty arrays

Nevertheless, I can understand a lot is going on in the command (I don't know), 
and don't say it is easy stuff
My only hope is that 4D will investigate the command.

And if nothing can be done, then I always can recode a couple of forms and use 
the tips mentioned by the iNUG community (you guys)

Gr,

Piotr

> -Oorspronkelijk bericht-
> Van: Chip Scheide [mailto:4d_o...@pghrepository.org]
> Verzonden: dinsdag 5 december 2017 15:59
> Aan: 4D iNug Technical <4d_tech@lists.4d.com>
> CC: Piotr Chabot Stadhouders <p.stadhoud...@timeff.com>
> Onderwerp: RE: poor performance LISTBOX INSERT COLUMN
> 
> if I am correct...
> list boxes should/do behave as arrays.
> 
> So... you should - no I have not done the testing - that inserting (add to 
> location
> { end).
> 
> EVEN IF, the array's memory did not have to be reallocated (it does), if you
> insert into location 10 of a 100 element array, 90 elements have to be 
> "shifted"
> to their current location +1 (10 -> 11, 11-> 12 etc).
> This will/does/should add overhead.
> 
> If you are inserting - especially blank/empty columns - rather then appending
> the columns to your list boxes I would expect this to be slower.  In a 
> previous
> email I mentioned that I do dynamic list boxes every where I use them.
> 
> I always (generic code) append the new column(s).
> Listbox Insert column(listbox;size of listbox+1; colname; etc)
> 
> 
> On Tue, 5 Dec 2017 09:11:25 +, Piotr Chabot Stadhouders via 4D_Tech
> wrote:
> >
> > I suppose you mean with the same command? Do you mean you think
> > putting the column behind the last could be faster than when putting
> > columns in front others?
> I would expect this to be the case.
> 
> ---
> Gas is for washing parts
> Alcohol is for drinkin'
> Nitromethane is for racing
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: poor performance LISTBOX INSERT COLUMN

[Chip Scheide via 4D_Tech]

if I am correct...
list boxes should/do behave as arrays.

So... you should - no I have not done the testing -
that inserting (add to location { 11, 11-> 12 etc).
This will/does/should add overhead.

If you are inserting - especially blank/empty columns - rather then 
appending the columns to your list boxes I would expect this to be 
slower.  In a previous email I mentioned that I do dynamic list boxes 
every where I use them.

I always (generic code) append the new column(s).
Listbox Insert column(listbox;size of listbox+1; colname; etc)

 
On Tue, 5 Dec 2017 09:11:25 +, Piotr Chabot Stadhouders via 4D_Tech 
wrote:
> 
> I suppose you mean with the same command? Do you mean you think 
> putting the column behind the last could be faster than when putting 
> columns in front others?
I would expect this to be the case.

---
Gas is for washing parts
Alcohol is for drinkin'
Nitromethane is for racing 
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: poor performance LISTBOX INSERT COLUMN

[Piotr Chabot Stadhouders via 4D_Tech]

Hi Arnaud,

Thanks for your thorough test.


>That's not surprising, IMHO, it's like concatenation: as the room is filled, 
>each piece added is harder to arrange than the previous. 

I understand this, but still I believe maybe there can be some performance 
improvement in the command. I cannot believe that after inserting 400 columns 
4D needs near to half a second to do this.



>each ctrl+D displays the wheel of death, each time longer than the previous. I 
>could never bear such waiting time!
->-> 4D appreciates much better duplicate at runtime. 

In my tests I also saw this happening. Duplicating each listbox in the form 
editor with 20 columns took longer and longer.
I didn't know / expected this to be faster in runtime.


Gr,

Piotr

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: poor performance LISTBOX INSERT COLUMN

[Arnaud de Montard via 4D_Tech]

> Le 5 déc. 2017 à 12:08, Arnaud de Montard via 4D_Tech <4d_tech@lists.4d.com> 
> a écrit :
> 
> --> 4D appreciates much better duplicate at runtime. 

PS
and the .4DB file [size + complexity] too, I guess. 

-- 
Arnaud de Montard 




**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: poor performance LISTBOX INSERT COLUMN

[Arnaud de Montard via 4D_Tech]

> Le 5 déc. 2017 à 10:11, Piotr Chabot Stadhouders via 4D_Tech 
> <4d_tech@lists.4d.com > a écrit :
> 
> LISTBOX INSERT COLUMN becomes slower when more columns have been inserted

That's not surprising, IMHO, it's like concatenation: as the room is filled, 
each piece added is harder to arrange than the previous. 

You should try a form with one "template" listbox having columns in excess, 
then on runtime [duplicate listbox / fill with data / remove useless columns], 
as much times as you need. Based on these tests: 

1/ template listbox has one column, I duplicate it 12 times and use LISTBOX 
INSERT COLUMN to each duplicate
this is what happens when the final number of colums grows (20, 40 ,60, etc.)
>
--> time to create columns is exponential… 

2/ template listbox has 100 columns
 2a/ duplicate it 12 times so we have 12 LB of 100 columns each at end
 345 ms (reference time)
 2b/ duplicate it 12 times, and delete 80 columns in each clone
 342 ms
--> time to delete columns is negligible

3/ I wanted to try with pre existing 12 listboxes of 100 columns in the form 
(intead of duplicating a template one at runtime). So I began in the form 
editor: select listbox, ctrl+D, ctrl+D, ctrl+D… 
 
each ctrl+D displays the wheel of death, each time longer than the previous. I 
could never bear such waiting time!
--> 4D appreciates much better duplicate at runtime. 

-- 
Arnaud de Montard 




**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: poor performance LISTBOX INSERT COLUMN

[Piotr Chabot Stadhouders via 4D_Tech]

hi Alan,

Maybe calling the average was a bit misleading after all :

So I created a demo database with 1 form
On this form I have 12 listboxes with 1 column
On load of each listbox I insert 20 columns with LISTBOX INSERT COLUMN

Inspecting the 4D debug log I am seeing something I did mention below :
LISTBOX INSERT COLUMN becomes slower when more columns have been inserted

The first call executes in 0,001 second
The last call (240) executes in 0,029 second

Do you think this is normal behavior, because more "objects" are on the form? 
Or is this maybe a performance issue?

By the way, what did you mean with " I assumed that you were "appending" column 
instead of inserting column before existing columns " ?
I suppose you mean with the same command? Do you mean you think putting the 
column behind the last could be faster than when putting columns in front 
others?

Gr,

Piotr


> -Oorspronkelijk bericht-
> Van: Alan Chan [mailto:a...@belhk.com]
> Verzonden: maandag 4 december 2017 9:46
> Aan: 4d_tech@lists.4d.com
> Onderwerp: Re: poor performance LISTBOX INSERT COLUMN
> 
> 4D iNug Technical <4d_tech@lists.4d.com> writes:
> >Maybe it does matter how many listboxes are on the form?
> 
> I assumed you built LB with one method per LB.
> >
> >Maybe LISTBOX INSERT COLUMN becomes slower when more columns are
> inserted?
> 
> I assumed that you were "appending" column instead of inserting column
> before existing columns.
> 
> Optimization is fun although some developers are against it.
> 
> Health warning : Optimization could be highly addictive and hazardous to your
> mentality.
> 
> Alan Chan
> 

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: poor performance LISTBOX INSERT COLUMN

[Piotr Chabot Stadhouders via 4D_Tech]

Hi Chip,

When the form is first displayed there are no elements in one of the arrays in 
the listboxes
Only after certain user action, like pressing a button, the arrays are 
populated, so this isn't the problem

Furthermore, I am not able to test with local 4D data because, surprise, we 
don't use the 4D datafile at all. We populate the arrays with Oracle data.

Gr,

Piotr


> maybe it is the remote data call/population of the listbox arrays?

> for comparison try:
> - populate the listbox from local (4D) data
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: poor performance LISTBOX INSERT COLUMN

[Chip Scheide via 4D_Tech]

Piotr,
maybe it is the remote data call/population of the listbox arrays?

for comparison try:
- populate the listbox from local (4D) data

On Mon, 4 Dec 2017 07:59:00 +, Piotr Chabot Stadhouders via 4D_Tech 
wrote:
> 
> I tested this on an HP Intel Core i7-4710MQ 2.50GHz laptop with 24G memory
> When showing the listboxes for the first time the arrays have 0 
> elements (after on load they are populated via 4D for OCI commands)
> Maybe it does matter how many listboxes are on the form?
> Maybe LISTBOX INSERT COLUMN becomes slower when more columns are inserted?
---
Gas is for washing parts
Alcohol is for drinkin'
Nitromethane is for racing 
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: poor performance LISTBOX INSERT COLUMN

[Alan Chan via 4D_Tech]

4D iNug Technical <4d_tech@lists.4d.com> writes:
>Maybe it does matter how many listboxes are on the form?

I assumed you built LB with one method per LB.
>
>Maybe LISTBOX INSERT COLUMN becomes slower when more columns are inserted?

I assumed that you were "appending" column instead of inserting column before 
existing columns.

Optimization is fun although some developers are against it.

Health warning : Optimization could be highly addictive and hazardous to your 
mentality.

Alan Chan

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: poor performance LISTBOX INSERT COLUMN

[Piotr Chabot Stadhouders via 4D_Tech]

Hi Alan,

Thanks for testing! This is great news, because then maybe there is something 
else going on.
Remember, I only asked the community if this performance was normal behavior 
and if it could be confirmed.
Clearly not by you.

I tested this on an HP Intel Core i7-4710MQ 2.50GHz laptop with 24G memory
When showing the listboxes for the first time the arrays have 0 elements (after 
on load they are populated via 4D for OCI commands)
Maybe it does matter how many listboxes are on the form?
Maybe LISTBOX INSERT COLUMN becomes slower when more columns are inserted?

Miyako explained clearly LISTBOX INSERT COLUMN is a UI element, and that there 
are graphic engine calls involved
I use a 1920x1080 resolution.

I think I am going to test the creation of 20 columns for 10 listboxes on a 
blank form and see what happens.

Again, Thanks for testing Alan

Gr,

Piotr

> I just tested LISTBOX INSERT COLUMN command on array based LB that takes only 
> 0 to 1 ms to complete on my MBP 2.90GHZ with 16G memory. I'm not sure if size 
> of array is the factor.


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: poor performance LISTBOX INSERT COLUMN

[Alan Chan via 4D_Tech]

I just tested LISTBOX INSERT COLUMN command on array based LB that takes only 0 
to 1 ms to complete on my MBP 2.90GHZ with 16G memory. I'm not sure if size of 
array is the factor.

Alan Chan

4D iNug Technical <4d_tech@lists.4d.com> writes:
>The LISTBOX INSERT COLUMN command takes on average 0.0286 seconds to complete, 
>tested on a Intel Core i7-4710MQ 2.50GHz laptop

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: poor performance LISTBOX INSERT COLUMN

[Alan Chan via 4D_Tech]

If your customer want to be king, he/she has to act like one, spend like one.

Did your customer try to put up more resources on hardware? Investing 32G+ 
memory on 4.0Ghz+ machine with great grahic chip for big monitor to run your 
application might be helpful especially populating 27 listbox with total 477 
columns (unknow
number of rows).

Alan Chan

4D iNug Technical <4d_tech@lists.4d.com> writes:
>When customers ask for showing 10 listboxes at once on screen, with each 
>listbox having 20 columns I don't know if it is reasonable, but as you 
>probably know customer is king.

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: poor performance LISTBOX INSERT COLUMN

[Keisuke Miyako via 4D_Tech]

thinking aloud, does it make any difference if you insert 400 empty arrays,
and then populate (copy array) the rows,
instead of inserting arrays that already contain data?

> 2017/12/03 19:08、Piotr Chabot Stadhouders via 4D_Tech <4d_tech@lists.4d.com> 
> のメール：
>
> We use listboxes a lot and we really like them (We use array listboxes by the 
> way, populated by an Oracle DB).
> I think 4D invests a lot of time in making listboxes more advanced, and they 
> succeed very well.
> But I don't agree that inserting 400 columns in a list is something really 
> spectacular.
> When this command isn't designed for this, maybe they can give me another 
> command that IS designed for this.




**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: poor performance LISTBOX INSERT COLUMN

[Piotr Chabot Stadhouders via 4D_Tech]

Hi Miyako,

>I think we need to set reasonable limits on how the commands is expected to be 
>used,
>before we device that its performance is "poor".

Sorry, maybe you are right and I shouldn't have called it poor performance, but 
it was my feeling because the command takes 10 times longer to execute than all 
other commands


>is it really reasonable to insert 400 columns in one call?

When customers ask for showing 10 listboxes at once on screen, with each 
listbox having 20 columns I don't know if it is reasonable, but as you probably 
know customer is king.
Even worse, customers want 20 columns per listbox, but than are making 60% 
invisible
So, I could do a lot to make things faster. Maybe, at first only create 
listboxes that are visible, and maybe even only create at first columns that 
are visible. But I prefer not to.


>at what point in developer did you notice the "poor performance"?

I must say I don't quite remember, because it has been a while. We first had 
the issue with some forms that were less important, and not so heavily used
But I am sure it began when converting from one 4D version to another
I believe it was when we converted our app from 4D v2004 to 4D v13


>the command name suggests that it is designed to insert a single column.
>so you can imagine that it goes through the same check list for every call.

So this sounds to me that a command LISTBOX INSERT COLUMNS would be a great 
addition, maybe with a width parameter at the same time


>but the speed issue discussed in this thread,
>seem to me like a case where a command has been used in a way for which it was 
>not designed.

We use listboxes a lot and we really like them (We use array listboxes by the 
way, populated by an Oracle DB).
I think 4D invests a lot of time in making listboxes more advanced, and they 
succeed very well.
But I don't agree that inserting 400 columns in a list is something really 
spectacular.
When this command isn't designed for this, maybe they can give me another 
command that IS designed for this.

 
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: poor performance LISTBOX INSERT COLUMN

[Keisuke Miyako via 4D_Tech]

in my view,
I think we need to set reasonable limits on how the commands is expected to be 
used,
before we device that its performance is "poor".

is it really reasonable to insert 400 columns in one call?
at what point in developer did you notice the "poor performance"?

did you gradually extend the size,
and suddenly the speed dropped?

are you working on a proof of concept to see whether the design is feasible,
or have you already decided that the design is totally sane and 4D must be 
broken?

the command name suggests that it is designed to insert a single column.
so you can imagine that it goes through the same check list for every call.
the listbox is a UI element, so there must be some graphic engine calls 
involved.
(font metric calculation, scrollbar calculation, etc)
you can imagine you force 4D to discard and redo 400 computations.

you must also consider that a form may have multiple pages in the form editor,
but to the OS, it is just one window with a lot of hidden objects.

the "page" concept is offered for the developer's convenience,
but the cost of delivery is that at runtime the window has a lot of baggage.

there are other ways to manage multiple listboxes in a single dialog window.
re-using the same object is one way.
switching the content of a subform is another.

there are ways to improve the UX.
deferring the content population with a "pragma once" timer is one way.
using a worker and CALL FORM is another.
postponing the setup until the box is displayed, is yet another.

the speed must be unreasonably slow in order to define "poor performance" as a 
bug.
for example, there was a recent bug (fixed) whether access to the pasteboard 
took up to 1 minute,
on High Sierra. now that is slow.

but the speed issue discussed in this thread,
seem to me like a case where a command has been used in a way for which it was 
not designed.





**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: poor performance LISTBOX INSERT COLUMN

[Tim Nevels via 4D_Tech]

On Dec 2, 2017, at 2:00 PM,Piotr Chabot Stadhouders wrote:

>> There is an old saying where a man goes to the doctor and says “when I move
>> my arm like this it hurts, can you fix this” and the doctor replies “yes, 
>> don’t
>> move your arm like that”. :)
> 
> One could also say : "A man goes to a car mechanic and says : I have a flat 
> tire. The mechanic says : Go walk” 

But there is one difference. When you go to the mechanic you are willing pay 
him to fix your tire. Most likely the mechanic will want your money and will 
gladly fix your tire. 

> Defining the listboxes in the form editor was just the way I did it in the 
> past.
> Also putting listboxes stacked on top of each other is something I am doing 
> frequently
> Because I had to modify the listboxes on a regular basis (add columns in 
> request of users) I decided to create them programmatically
> Why does 4D implement all these new (listbox) object commands? I think just 
> for this reason to create all things programmatically

This is obviously a command that needs some attention and most likely can be 
optimized to make it faster. Apparently in all the development and testing when 
creating this command nobody ever used it to the high level that you are and 
noticed the poor performance.

>> 1. Wait for 4D to make a change to fix this (who knows if/when that would
>> happen) 2. Don’t use LISTBOX INSERT COLUMN
> 
> I am going to wait a bit more, but finally, if nothing happens, I am afraid I 
> have to create al columns in the form editor again.

So you know the drill… Submit a bug report with all your details so that 4D 
knows there is a performance problem. Remember, talking about it here will not 
cause 4D to take any action.

Second, go to 4D Forums and create a feature request to speed up LISTBOX INSERT 
COLUMN. Show your examples of how slow it is. 

Wouldn’t it be nice if 4D offered a service — like the mechanic — where you 
could say, "what would you charge me to fix this 4D problem”. And if the price 
was right, you would pay them and you could have this problem fixed quickly. 
But, as we all know, that is not the world that we live in.

The only way to get something done is to get 4D Engineering’s attention so that 
they will assign someone to go work on this command and make it faster.

Tim


Tim Nevels
Innovative Solutions
785-749-3444
timnev...@mac.com


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: poor performance LISTBOX INSERT COLUMN

[Arnaud de Montard via 4D_Tech]

> Le 2 déc. 2017 à 10:10, Jim Dorrance via 4D_Tech <4d_tech@lists.4d.com> a 
> écrit :
> 
> Ever use:
> 
> $tSQL:="SELECT * FROM "+$tTableName+" INTO : "+$tLBox
> [...]

Hi Jim, 
I swear I did not copy/paste your answer   :-)

Makes me think I forgot to mention that one must check 4D version before using 
INTO LISTBOX with more than one LB: 

I've seen it's corrected recently in a v16, but can't remember exactly which 
one. See Frank Louvigne bug reference in the mentioned thread to check that. 

BTW… 
I don't like the way 4D gives names to columns and headers in those "SQL from" 
listboxes. 
I'd like an option "use LB name as root":
 LB name = "myLB"
 col1 = "myLB_c1"
 col2 = "myLB_c2"
 head1 = "myLB_h1"
 head2 = "myLB_h2"
etc. 

-- 
Arnaud de Montard 


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: poor performance LISTBOX INSERT COLUMN

[Arnaud de Montard via 4D_Tech]

> Le 2 déc. 2017 à 09:38, Piotr Chabot Stadhouders via 4D_Tech 
> <4d_tech@lists.4d.com> a écrit :
> 
> Hi Arnaud,
> 
> Array listboxes

Hi Piotr, 
thanks. These users are of the air controller kind  ;-) 

If the performance issue is strictly related to the command that adds a new 
column, i'm wondering if using pre sized listboxes with "too much" columns 
could fix it - you would just have to mask columns in excess. 

Do you (can you…) use SQL "SELECT ... INTO LISTBOX" for some of these 
listboxes? 

You can cheat, too: fill the minimum set of LB on load, then delegate to 
something else (timer and/or another process) the filling of the rest. The user 
may be right to want it all, but still he has 1 brain and 2 eyes. 

-- 
Arnaud de Montard 


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: poor performance LISTBOX INSERT COLUMN

[Jim Dorrance via 4D_Tech]

Ever use:

$tSQL:="SELECT * FROM "+$tTableName+" INTO : "+$tLBox


to create a listbox with all fields as columns and then hide the unwanted
columns?

On Sat, Dec 2, 2017 at 9:38 AM, Piotr Chabot Stadhouders via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> Hi Arnaud,
>
> Array listboxes
>
> >> Le 1 déc. 2017 à 15:49, Piotr Chabot Stadhouders via 4D_Tech
> <4d_t...@xxx.xxx> a écrit :
> >>
> >> Hi,
> >>
> >> In search of a performance problem in 1 of our forms I am asking for
> some advice.
> >>
> >> I have succeeded in analyzing 10 4DDebuglog.txt files (using an Oracle
> DB by the way) and have come to following :
> >> The problem is with the execution time of the "LISTBOX INSERT COLUMN"
>
> > array or selection listbox?
>
> > --
> > Arnaud
>
> **
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **
>



-- 
Jim Dorrance
jim.dorra...@gmail.com
4...@dorrance.eu
www.4d.dorrance.eu

PS: If you know of anyone that needs an experienced 4D programmer to add
energy and experience to their team, please let me know. I have
experience in many areas. Reasonable rates. Remote or Paris only.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: poor performance LISTBOX INSERT COLUMN

[Piotr Chabot Stadhouders via 4D_Tech]

Hi Arnaud,

Array listboxes

>> Le 1 déc. 2017 à 15:49, Piotr Chabot Stadhouders via 4D_Tech 
>> <4d_t...@xxx.xxx> a écrit :
>> 
>> Hi,
>> 
>> In search of a performance problem in 1 of our forms I am asking for some 
>> advice.
>> 
>> I have succeeded in analyzing 10 4DDebuglog.txt files (using an Oracle DB by 
>> the way) and have come to following :
>> The problem is with the execution time of the "LISTBOX INSERT COLUMN"

> array or selection listbox?

> -- 
> Arnaud

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: poor performance LISTBOX INSERT COLUMN

[Piotr Chabot Stadhouders via 4D_Tech]

Hi Tim,

> -Oorspronkelijk bericht-
> Van: Tim Nevels [mailto:timnev...@mac.com]
> Verzonden: vrijdag 1 december 2017 22:00
> Aan: 4d_tech@lists.4d.com
> Onderwerp: Re: poor performance LISTBOX INSERT COLUMN
> 
> On Dec 1, 2017, at 2:00 PM,Piotr Chabot Stadhouders wrote:
> 
> > But, bottom line, we are skipping the fact that still the command
> > LISTBOX INSERT COLUMN is very slow compared to all other commands And
> > we wouldn't probably having this discussion if it was executed as fast
> > as the other commands
> 
> There is an old saying where a man goes to the doctor and says “when I move
> my arm like this it hurts, can you fix this” and the doctor replies “yes, 
> don’t
> move your arm like that”. :)

One could also say : "A man goes to a car mechanic and says : I have a flat 
tire. The mechanic says : Go walk" 


> 
> Is there a reason you MUST call LISTBOX INSERT COLUMN or is it just the way
> you have designed the form and your coding style. I know a lot of people think
> it is the best way to program to build all your list boxes programmatically. 
> Do it
> all with code and they have a dozen methods that do all the work. It’s a
> programming style.
> 
> The other option is to define your list boxes with the form editor defining 
> all the
> columns and properties. Then when the form is loaded all that column
> definition and insertion is eliminated. That is my personal programming
> preference.
> 
> If some of your list boxes are used for several purposes you could also create
> multiple list boxes that are stacked on top of each other, each setup exactly
> how you need it. Then you show/hide the ones you need as appropriate.

Defining the listboxes in the form editor was just the way I did it in the past.
Also putting listboxes stacked on top of each other is something I am doing 
frequently
Because I had to modify the listboxes on a regular basis (add columns in 
request of users) I decided to create them programmatically
Why does 4D implement all these new (listbox) object commands? I think just for 
this reason to create all things programmatically


> 
> I know this is probably not what you want to hear, but if you are determined 
> to
> cut down on load and display time, and you have found LISTBOX INSERT
> COLUMN is causing issues, you only have 2 choices:
> 
> 1. Wait for 4D to make a change to fix this (who knows if/when that would
> happen) 2. Don’t use LISTBOX INSERT COLUMN
> 

I am going to wait a bit more, but finally, if nothing happens, I am afraid I 
have to create al columns in the form editor again.


> Tim
> 
> 
> Tim Nevels
> Innovative Solutions
> 785-749-3444
> timnev...@mac.com
> 
> 

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: poor performance LISTBOX INSERT COLUMN

[Arnaud de Montard via 4D_Tech]

> Le 1 déc. 2017 à 15:49, Piotr Chabot Stadhouders via 4D_Tech 
> <4d_tech@lists.4d.com> a écrit :
> 
> Hi,
> 
> In search of a performance problem in 1 of our forms I am asking for some 
> advice.
> 
> I have succeeded in analyzing 10 4DDebuglog.txt files (using an Oracle DB by 
> the way) and have come to following :
> The problem is with the execution time of the "LISTBOX INSERT COLUMN"

array or selection listbox?

-- 
Arnaud 



**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: poor performance LISTBOX INSERT COLUMN

[Tim Nevels via 4D_Tech]

On Dec 1, 2017, at 2:00 PM,Piotr Chabot Stadhouders wrote:

> But, bottom line, we are skipping the fact that still the command LISTBOX 
> INSERT COLUMN is very slow compared to all other commands
> And we wouldn't probably having this discussion if it was executed as fast as 
> the other commands

There is an old saying where a man goes to the doctor and says “when I move my 
arm like this it hurts, can you fix this” and the doctor replies “yes, don’t 
move your arm like that”. :)

Is there a reason you MUST call LISTBOX INSERT COLUMN or is it just the way you 
have designed the form and your coding style. I know a lot of people think it 
is the best way to program to build all your list boxes programmatically. Do it 
all with code and they have a dozen methods that do all the work. It’s a 
programming style.

The other option is to define your list boxes with the form editor defining all 
the columns and properties. Then when the form is loaded all that column 
definition and insertion is eliminated. That is my personal programming 
preference. 

If some of your list boxes are used for several purposes you could also create 
multiple list boxes that are stacked on top of each other, each setup exactly 
how you need it. Then you show/hide the ones you need as appropriate.

I know this is probably not what you want to hear, but if you are determined to 
cut down on load and display time, and you have found LISTBOX INSERT COLUMN is 
causing issues, you only have 2 choices:

1. Wait for 4D to make a change to fix this (who knows if/when that would 
happen)
2. Don’t use LISTBOX INSERT COLUMN

Tim


Tim Nevels
Innovative Solutions
785-749-3444
timnev...@mac.com


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: poor performance LISTBOX INSERT COLUMN

[Kirk Brooks via 4D_Tech]

Piotr,

On Fri, Dec 1, 2017 at 8:59 AM, Piotr Chabot Stadhouders via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> In fact our customers have big screens an want to see a lot of data at once
> They like how it is displayed right now.
> I have counted the listboxes displayed at once and the number is 10 !!
> So, after a lot of recoding they still have to wait for more than 5 seconds
>

​I may be missing the point but is it the data in the arrays changing? If
so you know you don't need to re-draw the listbox from scratch, just update
the arrays.


-- 
Kirk Brooks
San Francisco, CA
===

*The only thing necessary for the triumph of evil is for good men to do
nothing.*

*- Edmund Burke*
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: poor performance LISTBOX INSERT COLUMN

[Piotr Chabot Stadhouders via 4D_Tech]

Hi Chip,

Thanks for your quick response, it is much appreciated

First of all you are right that I could recode some parts and is also for other 
reasons sometimes no bad idea at all

> Load the list boxes as you go. Depending on your usage, you might only need 1,
> or 4 list boxes not 27(!)

In fact our customers have big screens an want to see a lot of data at once
They like how it is displayed right now.
I have counted the listboxes displayed at once and the number is 10 !!
So, after a lot of recoding they still have to wait for more than 5 seconds 


> When the form loads, the Tab control is used to determine what data to
> display, and the listbox is built appropriately

I know I could do something like this, and maybe there is no escaping of doing 
this
I also read about a way to create subforms and use OBJECT SET SUBFORM to 
dynamically change the subform


But, bottom line, we are skipping the fact that still the command LISTBOX 
INSERT COLUMN is very slow compared to all other commands
And we wouldn't probably having this discussion if it was executed as fast as 
the other commands


Again, thanks for your response,

Piotr


> -Oorspronkelijk bericht-
> Van: Chip Scheide [mailto:4d_o...@pghrepository.org]
> Verzonden: vrijdag 1 december 2017 16:35
> Aan: 4D iNug Technical <4d_tech@lists.4d.com>
> CC: Piotr Chabot Stadhouders <p.stadhoud...@timeff.com>
> Onderwerp: Re: poor performance LISTBOX INSERT COLUMN
> 
> Piotr
> 
> recode this!
> On Fri, 1 Dec 2017 14:49:11 +, Piotr Chabot Stadhouders via 4D_Tech
> wrote:
> >
> > No big deal you could say, but for a couple of our forms it IS a big
> > deal We have a form with 27 listboxes on it, all dynamically build,
> > with a total of 477 columns
> recode this - I can't say this enough!
> 
> 
> I do not know what your interface looks like, however, I find it difficult to
> believe that more then 2 or 4 list boxes are on screen at one time, maybe not
> even that.
> 
> Load the list boxes as you go. Depending on your usage, you might only need 1,
> or 4 list boxes not 27(!)
> 
> Basic concept (assume 1 listbox on screen at a time - modify to meet your
> specific on screen needs, the base form has ONE listbox, with no
> columns)
> On load
>  - determine what is being displayed (which table/data set)
>  - setup the listbox for this data
> 
> during execution of the form.
> repeat as needed
>  - user changes 'page', using a tab control (or whatever user does to change
> displayed data)
>- determine which data set is needed based on user 'request' (above
> step)
>- (re)build the listbox for this new data set - using the SAME listbox <-- 
> this is
> the key... the SAME LISTBOX
> 
> using this approach you should not need to build/load more list boxes then are
> currently on screen.
> In addition to this you can reduce the form's complexity by removing
> 20+ list boxes, this in it self will help the form load faster, and use
> a far smaller memory footprint.
> Reducing your form complexity may increase code complexity - but the
> increase will be a LOT less then the decrease in your form's complexity.
> 
> As example:
> - Listing forms : for my systems I use a single form/listbox (selection
> based) for record listings. It is built based on the current table to be 
> displayed.
> The base listbox has NO columns in it. The listing form is a project form, 
> and in
> a few instances is used as a parent form for inheritance (when additional
> functionality is needed in addition to the standard functions provided).
> 
> - Entry forms : many of the entry forms in my systems display related data 
> (i.e.
> Invoice [parent], and Invoice items [children in listbox]).
> Some forms have as many as 11 "pages" of related information to be displayed,
> controlled by a Tab Control (not actual form pages). These entry forms have 1
> listbox on page 0 (zero), so the form has 2 pages, page 0 (zero) and page 1.
> When the form loads, the Tab control is used to determine what data to
> display, and the listbox is built appropriately. During form execution, when 
> the
> user selects a specific Tab in the control, the SAME listbox is re-built to 
> display
> that specific information.
> 
> In both of the instances (examples) above - interpretedly, in C/S, there is no
> noticeable time required to build the list boxes, regardless of how fast the 
> user
> changes tabs.
> 
> If you want some example code on how to do this, I can provide an example
> database (v11 or v13 I forget which) Actually, if anyone wants the demo - feel
> free to ask off list.
> Note: the demo stores data for building the list boxes in 3 tables in the
> database, and note THIS IS NOT a compon

Re: poor performance LISTBOX INSERT COLUMN

[Chip Scheide via 4D_Tech]

Piotr

recode this!
On Fri, 1 Dec 2017 14:49:11 +, Piotr Chabot Stadhouders via 4D_Tech 
wrote:
> 
> No big deal you could say, but for a couple of our forms it IS a big deal
> We have a form with 27 listboxes on it, all dynamically build, with a 
> total of 477 columns
recode this - I can't say this enough!


I do not know what your interface looks like, however, I find it 
difficult to believe that more then 2 or 4 list boxes are on screen at 
one time, maybe not even that.

Load the list boxes as you go. Depending on your usage, you might only 
need 1, or 4 list boxes not 27(!)

Basic concept (assume 1 listbox on screen at a time - modify to meet 
your specific on screen needs, the base form has ONE listbox, with no 
columns)
On load 
 - determine what is being displayed (which table/data set)
 - setup the listbox for this data

during execution of the form.
repeat as needed
 - user changes 'page', using a tab control (or whatever user does to 
change displayed data)
   - determine which data set is needed based on user 'request' (above 
step) 
   - (re)build the listbox for this new data set - using the SAME 
listbox <-- this is the key... the SAME LISTBOX

using this approach you should not need to build/load more list boxes 
then are currently on screen.
In addition to this you can reduce the form's complexity by removing 
20+ list boxes, this in it self will help the form load faster, and use 
a far smaller memory footprint.
Reducing your form complexity may increase code complexity - but the 
increase will be a LOT less then the decrease in your form's complexity.

As example:
- Listing forms : for my systems I use a single form/listbox (selection 
based) for record listings. It is built based on the current table to 
be displayed. The base listbox has NO columns in it. The listing form 
is a project form, and in a few instances is used as a parent form for 
inheritance (when additional functionality is needed in addition to the 
standard functions provided). 

- Entry forms : many of the entry forms in my systems display related 
data (i.e. Invoice [parent], and Invoice items [children in listbox]). 
Some forms have as many as 11 "pages" of related information to be 
displayed, controlled by a Tab Control (not actual form pages). These 
entry forms have 1 listbox on page 0 (zero), so the form has 2 pages, 
page 0 (zero) and page 1.
When the form loads, the Tab control is used to determine what data to 
display, and the listbox is built appropriately. During form execution, 
when the user selects a specific Tab in the control, the SAME listbox 
is re-built to display that specific information.  

In both of the instances (examples) above - interpretedly, in C/S, 
there is no noticeable time required to build the list boxes, 
regardless of how fast the user changes tabs. 

If you want some example code on how to do this, I can provide an 
example database (v11 or v13 I forget which) 
Actually, if anyone wants the demo - feel free to ask off list.
Note: the demo stores data for building the list boxes in 3 tables in 
the database, and note THIS IS NOT a component, nor does it use 
'widgets' (subforms <-- really bad naming convention). The 3 tables 
are: Listbox setup data ([table], dimensions, listing or entry listbox 
etc), Column setup data (field or calculation info, order, formatting, 
enterable, visible etc), and query (specifics {fields, relations, field 
title to show users, etc} for the [table] displayed using an iTunes 
like search object)


---
Gas is for washing parts
Alcohol is for drinkin'
Nitromethane is for racing 
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

poor performance LISTBOX INSERT COLUMN

[Piotr Chabot Stadhouders via 4D_Tech]

Hi,

In search of a performance problem in 1 of our forms I am asking for some 
advice.

I have succeeded in analyzing 10 4DDebuglog.txt files (using an Oracle DB by 
the way) and have come to following :
The problem is with the execution time of the "LISTBOX INSERT COLUMN"
The performance of this command is very poor compared to other commands.
In fact, it takes about 10 times longer than every other 4D command in the logs

The LISTBOX INSERT COLUMN command takes on average 0.0286 seconds to complete, 
tested on a Intel Core i7-4710MQ 2.50GHz laptop
No big deal you could say, but for a couple of our forms it IS a big deal
We have a form with 27 listboxes on it, all dynamically build, with a total of 
477 columns

So 477 * 0.0286 seconds takes up 13.6422 seconds
This is all done in the On Load event so the user must wait terribly long for 
the form to show up on the screen

My question :
Do you think this performance of LISTBOX INSERT COLUMN is acceptable / normal 
behavior?
Can someone confirm this performance?
Is there something, other than putting listboxes on other pages, I can do about 
this?

Thanks in advance,

Piotr

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**