Re: 4D Remote hanging at 3am after upgrading to Hi Sierra. Need logging advice.

[Keisuke Miyako via 4D_Tech]

F.Y.I.

ACI0098967 fixes the App Nap slowdown which could apply to a 4D Server with no 
active user interface (i.e. Admin Window).

this is v17 build 230086 from 23rd Nov.

https://bugs.4d.fr/fixedbugslist?branch=17

for other versions I offer

https://github.com/miyako/4d-plugin-prevent-app-nap

> 2019/01/02 22:35、Tim Daniels via 4D_Tech <4d_tech@lists.4d.com>のメール:
>
> Under System Preferences --> Engergy Saver, it was set to turn the display
> off after three hours. I changed it to "Never." My theory is that when the
> screen is off it triggers some kind of different power management setting
> that is interfering with 4D. Fingers crossed!
>
> High Sierra 10.13.6
> 4D v17 build 17.225365 64-bit



**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra. Need logging advice.

[Tim Daniels via 4D_Tech]

For what it's worth, I have been having the same issue where on most nights
the backup never starts until I connect to the server via Screen Sharing.
In most cases, when I do connect the server crashes. When I reopen the
server the backup completes as normal.

Yesterday I changed a setting and the backup completed as normal last
night. I won't know for a few more days if it is actually fixed but I
thought I would tell you what I did in case you want to try it.

Under System Preferences --> Engergy Saver, it was set to turn the display
off after three hours. I changed it to "Never." My theory is that when the
screen is off it triggers some kind of different power management setting
that is interfering with 4D. Fingers crossed!

High Sierra 10.13.6
4D v17 build 17.225365 64-bit
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra. Need logging advice.

[Perkins, Bradley D via 4D_Tech]

The good news is that the problem hasn't occurred the last 2 nights.
I haven't had time to check if the backups didn't run though. It is 
possible that the database didn't change over the weekend and backups
were skipped. Friday was a federal holiday. Now that people are back at 
work I'll be surprised if we don't hang tomorrow at ~5 am. Fingers crossed.

Tim Nevels wrote:

> I’ve been following this thread and thought I’d post a comment. I too
> was confused and thought the problem was happening on 4D Server and it
> was running on a Mac with High Sierra. I think Miyako may have thought
> that too. That’s why he talked about purge.

FWIW, I manually ran purge on the client machine just in case it might
help.

> Since 4D Server is on Windows I don’t think it is a server problem
> unless it is a network layer problem. I don’t recall what 4D version
> you are running. But are you using the new network or “legacy” network
> layer?  I would try switching and see if that makes a difference.

We're running 4D 15.4. Still using the legacy network layer. I don't read 
anything
good about using the new layer, so I'll stay where we are.
Unless this is a problem that is known and fixed in 15.5 I'm not going 
to take the time to test and upgrade. I'm basically dealing with this problem 
for them for free. I first asked here hoping someone more familiar with High 
Sierra might point to a known issue that is responsible.

What is weird is that there is a correlation between the time the backup
runs and the time that the client hangs. In 20+ years of working with 4D
Server we haven't experienced that. Not in my recent memory at least.

> There have been reports of certain versions of 4D having issues on
> certain Mac models running High Sierra. Looks like you’ve run into one
> of those situations. So what are your options?

Where are those reports? What are the specific issues? This is why I
originally asked here. Are there workarounds?

> 1. Live with it. Obviously not really an option. 

It's not, but this incident and the time and expense we go through 
every time we have to do a major 4D upgrade just so that we can continue 
to keep using a legacy 4D application has the customer strongly considering 
dumping 4D after almost 23 years. The only reason they still use 4D is
because the solution has served them well. 

> 2. Switch 4D versions and hope it is a bug that got fixed and a new 
> version solves the problem. 

Upgrading to 16 isn't an option at this time. See 1 above. 

> 3. It is a weird machine specific problem. Wipe the machine,
> reinstall fresh High Sierra macOS and fresh 4D Remote version. 
> Maybe it fixes this machine problem. 

I've considered this as a last resort. This is a relatively new machine
that was installed when we did our v15 upgrade. It only has one OS upgrade. 
Rebuilding the machine is a much larger job than one would expect as I have to
install a bunch of additional software to be compliant with institutional 
cybersecurity requirements, reconfigure 3 Apache virtual hosts on top of setting
up client and all of the scripts I have in place to run client 24/7/365.
It is an 8-12 hour job at a minimum and would have to be done over a weekend.

> 4. Work around the problem.

Which is basically what I've done for 25 years of 4D development including
4D web serving on client since 95. My workarounds have been to rely on 
external tools that monitor 4D and keep it running. The problem now is that
those tools fail. They've been in place since we moved our Mac clients to
OS X (2004?). I've had to modify them over time, but they've always 
worked.

> Write an AppleScript that you launch just before the backup starts.
> Use LEP to do this. Then QUIT 4D. The script will pause for enough
> time for the backup to complete. Maybe 5 minutes or whatever it is for
> your situation. Then the script launches 4D Remote again and
> reconnects automatically to 4D Server. The script quits.

I already have an Applescript that will restart 4D client if not running. 
It relies on launchd though. It runs 24/7.
I would need to modify the structure and run a process that will quit the 
client before the backup runs. 
I'll also have to modify all of my keep alive scripts to be aware of the backup 
'window' 
so that they don't run during that time. 
I'd prefer to not have to do this unless absolutely necessary.

> If you can’t fix a problem, just avoid it and work around it. Would
> this be an option?

As mentioned we already do to some extent, but in this case I'd like to
solve the problem. Right now the workaround is to manually reboot the client
each morning. We support a global user base and even in the US I have east
coast web users whose day starts 3 hours earlier than mine so we've got cases
now where people can't use the system. I've actually been getting up early to 
check on and fix the problem the past few weeks. I can't keep doing that. 

---

Also, I saw Jim Crate's suggestion to have a script take a screen 

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra. Need logging advice.

[Tim Nevels via 4D_Tech]

On Nov 12, 2017, at 1:00 PM, 4d_tech-requ...@lists.4d.com wrote:

> The 4D Remote which is hanging is a Mac which was recently upgraded to
> High Sierra, but the backup is running on a Windows 2008r2 Server.
> The hang times are consistent with when the backup is running. First it
> was 3am. I changed the backup to run at 5am and not the hang happens at
> 5am. This only started happening after the client machine was upgraded to
> High Sierra.
> The backup is standard backup, with no running code, set via the
> configuration panel on 4D Server.
> 
> It isn't clear if or why I would want to to run purge from On Backup
> Shutdown? Does that also run on the client?

I’ve been following this thread and thought I’d post a comment. I too was 
confused and thought the problem was happening on 4D Server and it was running 
on a Mac with High Sierra. I think Miyako may have thought that too. That’s why 
he talked about purge. 

Since 4D Server is on Windows I don’t think it is a server problem unless it is 
a network layer problem. I don’t recall what 4D version you are running. But 
are you using the new network or “legacy” network layer?  I would try switching 
and see if that makes a difference. 

There have been reports of certain versions of 4D having issues on certain Mac 
models running High Sierra. Looks like you’ve run into one of those situations. 
So what are your options?

1. Live with it. Obviously not really an option. 
2. Switch 4D versions and hope it is a bug that got fixed and a new version 
solves the problem. 
3. It is a weird machine specific problem. Wipe the machine, reinstall fresh 
High Sierra macOS and fresh 4D Remote version. Maybe it fixes this machine 
problem. 
4. Work around the problem.

Write an AppleScript that you launch just before the backup starts. Use LEP to 
do this. Then QUIT 4D. The script will pause for enough time for the backup to 
complete. Maybe 5 minutes or whatever it is for your situation. Then the script 
launches 4D Remote again and reconnects automatically to 4D Server. The script 
quits. 

If you can’t fix a problem, just avoid it and work around it. Would this be an 
option?

Tim

Sent from my iPad
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra. Need logging advice.

[Jim Crate via 4D_Tech]

> On Nov 11, 2017, at 18:00, Perkins, Bradley D via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
> The hang times are consistent with when the backup is running. First it
> was 3am. I changed the backup to run at 5am and not the hang happens at
> 5am. This only started happening after the client machine was upgraded to
> High Sierra.
> The backup is standard backup, with no running code, set via the
> configuration panel on 4D Server.

If the client is getting disconnected when the backup runs, you may want to 
take a screenshot when the app becomes unresponsive to see if some alert is 
displayed. The script attempting to kill the app may be dismissing alerts. 
Supposedly there is a “screencapture” command so you could maybe do that in the 
script before killall. 

Jim Crate

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra. Need logging advice.

[Keisuke Miyako via 4D_Tech]

The thinking behind this recommendation, as I understand it, goes like this:

macOS has its own cache system.
backup creates a large file.
the system cache the large file to improve access speed the next time (which 
typically never happens, by the way).
free system memory runs out.
4D Server asks for some memory.
the system does all sorts of things (compression, paging, whatever) to "create" 
free memory. this takes time.
4D Server is waiting.
4D Client disconnects.

"purge" (which is not installed by default and requires sudo) is a developer 
tool that forced the system to let go of cached files.

so to answer your question, the backup database method does run on the server, 
not client,
but that does not mean that it has no effect on the performance of connected 
clients.

> 2017/11/12 5:56、Jim Crate via 4D_Tech <4d_tech@lists.4d.com> のメール：
>
> Do the backup database methods run on connected clients (4D Remote) as well 
> as on the server?
>
> Brad said the client is disconnecting or hanging when the server runs backup. 
> Even if the server could run purge after finishing the backup, how would that 
> help the remote client which had its connection interrupted?
>
> Jim Crate




**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra. Need logging advice.

[Jim Crate via 4D_Tech]

> On Nov 11, 2017, at 11:26, Timothy Penner via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> Is this a Mac?
> 
> If so, have you tried running a PURGE from the OS in On Backup Shutdown?
> 
> http://kb.4d.com/assetid=77831
> http://kb.4d.com/assetid=75978
> 
> -Tim

Do the backup database methods run on connected clients (4D Remote) as well as 
on the server?

Brad said the client is disconnecting or hanging when the server runs backup. 
Even if the server could run purge after finishing the backup, how would that 
help the remote client which had its connection interrupted?

Jim Crate

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra. Need logging advice.

[Timothy Penner via 4D_Tech]

Is this a Mac?

If so, have you tried running a PURGE from the OS in On Backup Shutdown?

http://kb.4d.com/assetid=77831
http://kb.4d.com/assetid=75978

-Tim

Sent from my Verizon, Samsung Galaxy smartphone


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra. Need logging advice.

[John DeSoi via 4D_Tech]

You may have said so earlier, but is the machine set to never sleep in the 
Energy control panel?

RE: caffeinate

If you launch 4D using caffeinate /path/to/4d, the machine should not go to 
sleep until 4D quits. See also the -w option where you can have 4D set it up 
after it has started.

John DeSoi, Ph.D.





> On Nov 5, 2017, at 3:25 PM, Perkins, Bradley D via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
>   Beyond all of the above, I would like to find out why this problem
> repeatedly occurs at 3am. If you've followed my earlier posts I do have a
> launchd job that runs at that time, but rescheduling it earlier did not
> move the problem to the earlier time. I even turned that job off for one
> night and that didn't prevent the 3am sleep. I grep'd all of my logs and
> could not find anything notable happening between 2:57 and 3:01.

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra. Need logging advice.

[Jim Crate via 4D_Tech]

On Nov 5, 2017, at 4:25 PM, Perkins, Bradley D via 4D_Tech 
<4d_tech@lists.4d.com> wrote:

> saw no value in running Apple's Server App. Even under OS X Server, I
> always ran Apache manually. I found that the Mac OS X Server's UI for
> managing web sites would often overwrite our custom configurations, e.g.,
> rewrite rules. I wasn't clear earlier but the 4D web server (Active4D)
> sits behind Apache and serves on 127.0.0.1 on a high port. All requests go
> through Apache.

If you install Server app, you’ll want to look at creating a webapp plist. It’s 
pretty simple and allows you to hook in your additional configuration without 
modifying Server-created files that will be overwritten.

> many as the on-machine script notifications. However, I have the external
> monitor to stop sending notifications after a few failures. Actually,
> given that the machine is sending mail every 3 minutes from 3:01 am until
> I restart 4D it isn't totally asleep.

This would seem to indicate it isn’t related to OS sleep (i.e. things 
controllable with pmset). It could possibly be related to App Nap. Does 4D v15 
properl support App Nap? I’m not sure if there is a way to turn it off 
system-wide. 

>   I think that this is a big part of the problem. Granted this has run
> reliably on past OS versions but I need to thoroughly retest this script.
> - It runs "killall 4D", but could be modified to kill the process. I just
> verified that  "killall 4D" worked at this time. After doing so, the
> auto-launch kicked in and restarted 4D.

The man page says the default signal sent by killall is TERM (15). You can see 
all signals by running “man signal”, but the short answer is that TERM will ask 
4D to quit, and KILL (9) will tell the OS to kill the process (basically a 
force quit).


>   Beyond all of the above, I would like to find out why this problem
> repeatedly occurs at 3am. If you've followed my earlier posts I do have a
> launchd job that runs at that time, but rescheduling it earlier did not
> move the problem to the earlier time. I even turned that job off for one
> night and that didn't prevent the 3am sleep. I grep'd all of my logs and
> could not find anything notable happening between 2:57 and 3:01.

But not every night at 3am, right? Is there consistency on the days of the 
week? Days of the month? 

This is a 4D client, maybe something is happening to the server or network at 
that time and the 4D client is losing it’s connection to the server?

Jim Crate

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra. Need logging advice.

[Keisuke Miyako via 4D_Tech]

typo, s if for system, not display.

enter "man caffeinate" in terminal for details.

> 2017/11/06 5:00、Keisuke Miyako via 4D_Tech <4d_tech@lists.4d.com> のメール：
> sprevent display from sleep (but not if running on battery)



**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra. Need logging advice.

[Keisuke Miyako via 4D_Tech]

Hello,

I am an avid user of caffeinate (it's a great name, by the way),
to keep my mac awake.

there are several modes (assertions) to pick and choose,

caffeinate -dimu is the most aggressive,

dprevent display from sleep
iprevent system from sleep
sprevent display from sleep (but not if running on battery)
mprevent disk from sleep
ufake user activity

typically I use it while projecting a presentation,
or when several time consuming downloads are going on and I leave the room,
for example.

> 2017/11/05 4:32、Perkins, Bradley D via 4D_Tech <4d_tech@lists.4d.com> のメール：
>
> . In researching this problem I also discovered the 'caffeinate' command. 
> This is a command-line system utility that is supposed to prevent the 
> computer from sleeping. I haven't tried it yet. Has anyone here used it? If 
> so, can you describe how?




**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra. Need logging advice.

[Jim Crate via 4D_Tech]

On Nov 4, 2017, at 3:32 PM, Perkins, Bradley D via 4D_Tech 
<4d_tech@lists.4d.com> wrote:

> I believe I've done everything I can using the pmset command to prevent 
> sleep, standby etc. It isn't clear if you can totally turn power management 
> off everywhere though. In researching this problem I also discovered the 
> 'caffeinate' command. This is a command-line system utility that is supposed 
> to prevent the computer from sleeping. I haven't tried it yet. Has anyone 
> here used it? If so, can you describe how?

I don’t know if this is possible in your situation, but for the Mac Mini 
running 4D Server on 10.12 for my client I installed Apple’s Server app after 
having a couple issues suspected to be related to the power management. We had 
already purchased Server for other machines. If you go this route, you will 
probably have to configure your Apache web serving through the Server app. If 
you leave the FTP services turned off it shouldn’t interfere with whatever 
you’re using to run the SFTP server.

> I first deployed this system in 1995 under 3.x and it has provided continuous 
> near 24/7/365 service since then. It has a small, but global user base which 
> is primarily web-based. The machine in question is the web 'client', but it 
> also runs a Apache for static web content and an SFTP server. This system has 
> remained largely unchanged functionality-wise in the last 10 years. The main 
> structure changes have been those required to upgrade major 4D versions. I 
> maintain this system, but haven't implemented many newer 4D features. I've 
> tried to stay aware of how 4D has changed.

Is Apache still serving static content when 4D Remote stops working? 

> 2. Continuously monitor the 4D web server. Every few minutes an off-site 
> monitoring agent requests a simple dynamically generated page. If the request 
> times out, or if the response contains unexpected content, I get notified. 
> These requests provide an added benefit of keeping 4D busy during light or no 
> usage periods.

You’ve only mentioned being notified by the scripts on the machine itself. Does 
the external service monitor also notify you that the 4D web server has stopped 
serving?

> 3. There is not an easy way to externally determine if 4D, or any Mac OS 
> desktop app has hung. You can easily detect if 4D is running, but not when it 
> is non-responsive. For us this is when 4D has encountered a low-level error 
> that can't be gracefully handled with an error handler. It doesn't help that 
> any displayed dialogs provide little useful information. One solution is to 
> have 4D run a process that writes a 'heartbeat' file every minute and have a 
> system process check for changes to this file. If the file is present and 
> hasn't changed in more than a few minutes, we'll assume that 4D has hung and 
> run a script to kill it. At this point the auto-launcher from (1) above takes 
> over. This is where we seem to be failing currently. 
> 
> If others have done (3) how do you implement your heartbeat? I write a 
> current timestamp into the file every minute. That assumes the file exists 
> and is writable. It usually is. Is there a better way that is more robust?

Which part of #3 is failing? Is the script to kill 4D being run and failing? Is 
it running “kill -9 process_id”? Is the auto-launch failing? Is the heartbeat 
file writing stopping while the service is actually running? There are numerous 
reports of problems with file writing and delayed processes. If only the 
heartbeat file writing is stopping, does the file actually exist and is still 
writable by that process? Where is the file located?

Does the external service monitor have an API? If so, could you periodically 
check the external service monitor for your service, and bounce 4D when there 
is actually a problem with the service? 
 

Jim Crate

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra. Need logging advice.

[Perkins, Bradley D via 4D_Tech]

This problem went away for a few days, but restarted a couple of nights ago. It 
still happens at 3am. I have not been able to observe what happens at that time 
and suspect that if I try I won't see the problem. I suspect that Apple's 
evolving power management features are part of the problem and that if a user 
were interacting with the system at 3am, we wouldn't see this because the 
system would not be idle. Spending the night on site, which is an hour away, 
isn't an option, nor is getting up in the middle of the night and remotely 
logging in in the hope I can observe an obvious problem.

I believe I've done everything I can using the pmset command to prevent sleep, 
standby etc. It isn't clear if you can totally turn power management off 
everywhere though. In researching this problem I also discovered the 
'caffeinate' command. This is a command-line system utility that is supposed to 
prevent the computer from sleeping. I haven't tried it yet. Has anyone here 
used it? If so, can you describe how?

I first deployed this system in 1995 under 3.x and it has provided continuous 
near 24/7/365 service since then. It has a small, but global user base which is 
primarily web-based. The machine in question is the web 'client', but it also 
runs a Apache for static web content and an SFTP server. This system has 
remained largely unchanged functionality-wise in the last 10 years. The main 
structure changes have been those required to upgrade major 4D versions. I 
maintain this system, but haven't implemented many newer 4D features. I've 
tried to stay aware of how 4D has changed.

While I can appreciate Apple's intentions with the power management features, 
my experience has been that they increasingly work against running 4D 24/7. 
This system is deployed on a network at a govt. agency. Their cyber security 
people force us to run the latest OS. No recent OS upgrade seems to go smoothly 
4D-wise. The 4D client in question had always run for weeks or even months w/o 
problems, but every Mac OS upgrade seems to lessen that. Moving this web system 
to Windows is not an option. We do run 4D Server on Windows.

I have a number of controls in place to to ensure that the system is running. 

1. Auto-launch 4D Remote in web serving mode if it is not running.  I can 
toggle this on/off. This is done with a combination of shell scripts and 
AppleScript. Apple's Launchd system manages this.
2. Continuously monitor the 4D web server. Every few minutes an off-site 
monitoring agent requests a simple dynamically generated page. If the request 
times out, or if the response contains unexpected content, I get notified. 
These requests provide an added benefit of keeping 4D busy during light or no 
usage periods.
3. There is not an easy way to externally determine if 4D, or any Mac OS 
desktop app has hung. You can easily detect if 4D is running, but not when it 
is non-responsive. For us this is when 4D has encountered a low-level error 
that can't be gracefully handled with an error handler. It doesn't help that 
any displayed dialogs provide little useful information. One solution is to 
have 4D run a process that writes a 'heartbeat' file every minute and have a 
system process check for changes to this file. If the file is present and 
hasn't changed in more than a few minutes, we'll assume that 4D has hung and 
run a script to kill it. At this point the auto-launcher from (1) above takes 
over. This is where we seem to be failing currently. 

If others have done (3) how do you implement your heartbeat? I write a current 
timestamp into the file every minute. That assumes the file exists and is 
writable. It usually is. Is there a better way that is more robust?

I know that 4D has improved logging features, but I've never used them. What 
would I need to do to enable 4D to log between 2:45 and 3:15 AM? 

Thanks,

Brad


From: Perkins, Bradley D
Sent: Monday, October 30, 2017 9:08 AM
To: 4d_tech@lists.4d.com
Subject: Re: 4D Remote hanging at 3am after upgrading to Hi Sierra.

I think I found the problem. When I upgraded the server OS on Tuesday I had a 
second GUI login session open for the admin user as I needed to sudo to fix 
some permissions, etc (aka fast user switching). When I checked on the 
application early Saturday AM I realized I had left that user logged in with an 
idle terminal session. I also found that when I remotely logged in to to check 
on the 'web client' user that 4D was not running, but quickly launched (my 
login woke the system up). I suspect that power management features were 
causing the network and/or hard disks to 'sleep' at 3AM. We have not had 
problems since logging that user out. Therefore, even though you may think that 
Power Nap is configured to be off, it isn't totally.

Brad Perkins.

On 10/27/17, 3:02 PM, "Perkins, Bradley D" <bperk...@lanl.gov> wrote:

I have grep'd everything in /var/log --- the files the the 

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra.

[Perkins, Bradley D via 4D_Tech]

I think I found the problem. When I upgraded the server OS on Tuesday I had a 
second GUI login session open for the admin user as I needed to sudo to fix 
some permissions, etc (aka fast user switching). When I checked on the 
application early Saturday AM I realized I had left that user logged in with an 
idle terminal session. I also found that when I remotely logged in to to check 
on the 'web client' user that 4D was not running, but quickly launched (my 
login woke the system up). I suspect that power management features were 
causing the network and/or hard disks to 'sleep' at 3AM. We have not had 
problems since logging that user out. Therefore, even though you may think that 
Power Nap is configured to be off, it isn't totally.

Brad Perkins.

On 10/27/17, 3:02 PM, "Perkins, Bradley D"  wrote:

I have grep'd everything in /var/log --- the files the the console displays 
-- and haven't found anything that is running at 3am. That is assuming of 
course that any system level activity would be logged. The closest entry is at 
3:01 and that is when my monitoring script emails me the first of many messages 
that there is a problem.

I was mainly asking here in case those that are more in tune with current 
Mac goings on were aware of anything new in High Sierra that might have caused 
problems with 4D.

I'm not familiar with the use of  4D's debug/request log. Can that be 
turned on easily, or do I have to recompile the structure to enable it?
I'm mainly interested in what is going on at the client. I'd also guess 
that whatever is hanging 4D might not get logged to the server.

Thanks,

Brad

> From: JBellos 

> Try turning on the debug/request log in 4D to see what's firing in your
> application at that time, if anything..

> Also, look at the Mac Console log to see what's happening at 3 AM, if
> there's anything unique running there that's not  happening earlier.

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra.

[Jim Medlen via 4D_Tech]

One thing you might try is changing the time that the script runs.

If the the script runs successfully at 2:00 AM and the computer hangs at
3:00 AM
then the problem is probably not the script.

If the computer hangs at the new scheduled time for the script and not
3:00AM
then that would lead me to look closer at the script.

We have had issues with writing files to folders where the permissions
would not
allow 4D to access a file in Read Write unless we changed the owner or
permissions
on the folder.

Good Luck.



**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: 4D Remote hanging at 3am after upgrading to Hi Sierra.

[Epperlein, Lutz (agendo) via 4D_Tech]

Brad, you wrote in your first posting, you saw a 4D error message very shortly. 
But you weren't able to read it. 
So it would be really helpful if a 4D Server would not show such messages only, 
they should also be logged into a file (or in system log) regardless of my 
settings, at least during the starting phase of the server.
The usual message in the (Windows) Event log that 4D is started is not really 
helpful.

I just filed a feature request, look here: 
http://forums.4d.com/Post//21398249/1/

This request doesn't help you so much in the present situation, but if this 
feature would exist already it would be really helpful and you would a step 
further in your problem solving, I think.


Thanks
Lutz Epperlein

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra.

[Perkins, Bradley D via 4D_Tech]

I have grep'd everything in /var/log --- the files the the console displays -- 
and haven't found anything that is running at 3am. That is assuming of course 
that any system level activity would be logged. The closest entry is at 3:01 
and that is when my monitoring script emails me the first of many messages that 
there is a problem.

I was mainly asking here in case those that are more in tune with current Mac 
goings on were aware of anything new in High Sierra that might have caused 
problems with 4D.

I'm not familiar with the use of  4D's debug/request log. Can that be turned on 
easily, or do I have to recompile the structure to enable it?
I'm mainly interested in what is going on at the client. I'd also guess that 
whatever is hanging 4D might not get logged to the server.

Thanks,

Brad

> From: JBellos 

> Try turning on the debug/request log in 4D to see what's firing in your
> application at that time, if anything..

> Also, look at the Mac Console log to see what's happening at 3 AM, if
> there's anything unique running there that's not  happening earlier.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra.

[Perkins, Bradley D via 4D_Tech]

Chip, 

/var/log/daily.out exists
Since the OS  is newly updated no weekly or monthlies have run yet.
I can see that the daily OS X cleanup scripts are running at ~22:15 local time. 
That rules these out.

Thanks,

Brad Perkins

On 10/27/17, 12:25 PM, "Chip Scheide" <4d_o...@pghrepository.org> wrote:

here is a link about the cron jobs/scripts
http://thexlab.com/faqs/maintscripts.html

I do not really know what they do, only that they exist, I only quickly 
scanned the article.
the article at the above link says they (may) run at 3:15am

of course, the underlying OS changed (Sierra to High Sierra),
existing anti-virus, or other things may interact differently - 
especially if you did not update that software with the OS.

EX: - 
where I work they run/use Sophos AV
under the OS we were running (10.6 - due to legacy applications), 
Sophos thinks - maybe rightly so :) - that MS Office is a virus and 
will not let it run. So we have had to get a different AV package 
(Intego Virus Barrier)
 

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra.

[Chip Scheide via 4D_Tech]

here is a link about the cron jobs/scripts
http://thexlab.com/faqs/maintscripts.html

I do not really know what they do, only that they exist, I only quickly 
scanned the article.
the article at the above link says they (may) run at 3:15am

of course, the underlying OS changed (Sierra to High Sierra),
existing anti-virus, or other things may interact differently - 
especially if you did not update that software with the OS.

EX: - 
where I work they run/use Sophos AV
under the OS we were running (10.6 - due to legacy applications), 
Sophos thinks - maybe rightly so :) - that MS Office is a virus and 
will not let it run. So we have had to get a different AV package 
(Intego Virus Barrier)


 
On Fri, 27 Oct 2017 16:46:59 +, Perkins, Bradley D wrote:
> Chip,
> 
> Except for the OS Upgrade nothing changed. This problem didn't exist 
> under Sierra.
> The machine has  AV installed (IT policy mandates that too) but it 
> has never been a problem in the past.
> I'm the only end user of the machine. As noted I do have a 3am job, 
> but that has run fine in the past and doesn't cause problems when 
> manually executed while 4D is running.
> What are the 'daily cleanup' routines that OS X runs? I'm assuming 
> previous OS X versions did that too, but want to look into this.
> 
> Thanks,
> 
> Brad
> 
> On 10/27/17, 10:38 AM, "Chip Scheide" <4d_o...@pghrepository.org> wrote:
> 
> since it is at the same time -
> There are a couple of possibilities:
> - anti virus software (either update, or scan)
> - cron job - specifically the daily 'cleanup' routines run by OS X
> - some other specific timed action setup by(?) IT, the end user(s)?
> 
> Chip
>  
> 
---
Gas is for washing parts
Alcohol is for drinkin'
Nitromethane is for racing 
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra.

[JBellos via 4D_Tech]

Try turning on the debug/request log in 4D to see what's firing in your
application at that time, if anything..

Also, look at the Mac Console log to see what's happening at 3 AM, if
there's anything unique running there that's not  happening earlier.



--
Sent from: http://4d.1045681.n5.nabble.com/4D-Tech-f1376241.html
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra.

[Perkins, Bradley D via 4D_Tech]

Chip,

Except for the OS Upgrade nothing changed. This problem didn't exist under 
Sierra.
The machine has  AV installed (IT policy mandates that too) but it has never 
been a problem in the past.
I'm the only end user of the machine. As noted I do have a 3am job, but that 
has run fine in the past and doesn't cause problems when manually executed 
while 4D is running.
What are the 'daily cleanup' routines that OS X runs? I'm assuming previous OS 
X versions did that too, but want to look into this.

Thanks,

Brad

On 10/27/17, 10:38 AM, "Chip Scheide" <4d_o...@pghrepository.org> wrote:

since it is at the same time -
There are a couple of possibilities:
- anti virus software (either update, or scan)
- cron job - specifically the daily 'cleanup' routines run by OS X
- some other specific timed action setup by(?) IT, the end user(s)?

Chip
 

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Remote hanging at 3am after upgrading to Hi Sierra.

[Chip Scheide via 4D_Tech]

since it is at the same time -
There are a couple of possibilities:
- anti virus software (either update, or scan)
- cron job - specifically the daily 'cleanup' routines run by OS X
- some other specific timed action setup by(?) IT, the end user(s)?

Chip

On Fri, 27 Oct 2017 16:22:36 +, Perkins, Bradley D via 4D_Tech 
wrote:
> We had to update a machine to Mac OS X Hi Sierra this week 
> (cybersecurity mandate) on Tuesday of this week.
> This machine runs as a web 'client' on 4D 15.4.
> I have long standing controls in place to monitor the 4D application 
> and keep it running.
> Since the OS update, 4D is hanging at 3am every morning and is not 
> recovering. I know this because my scripts are sending me email every 
> 3 minutes that there is a problem. I have to manually reboot 4D every 
> morning.
> 
> From what I can tell Power Nap is off on the machine:
> 
> $ pmset -g
> System-wide power settings:
> Currently in use:
> standby  1
> Sleep On Power Button 1
> womp 1
> autorestart  1
> hibernatefile/var/vm/sleepimage
> powernap 0
> gpuswitch2
> networkoversleep 0
> disksleep0
> sleep0 (sleep prevented by screensharingd)
> autopoweroffdelay28800
> hibernatemode0
> autopoweroff 1
> ttyskeepawake1
> displaysleep 30
> standbydelay 10800
> I have an external job that runs at 3am via launchd, but turned it 
> off yesterday to see if it was causing the problem. We still had the 
> problem this morning. I can also run that job manually while 4D is 
> running and it doesn't cause problems.
> When I accessed 4D this morning to reboot, I briefly saw a 4D error 
> dialog that would make me think the source of the problem is in a 
> 4Dprocess that writes a file to disk. That process is running now 
> without problem. Unfortunately the dialog dismissed itself before I 
> could write down details. Therefore, I suspect something is running 
> at ~3 that is not letting 4D write that file.
> 
> I've dug through logs and looked at other system settings to see if 
> the system is running something at that time, but haven't found 
> anything.
> 
> Any ideas would be appreciated?
> 
> -- Brad
> 
> 
> 
> **
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **
---
Gas is for washing parts
Alcohol is for drinkin'
Nitromethane is for racing 
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**