Derby Clientdriver and windows server (exchange)
I thought I had the problems with networked derby handled. But no... I've managed to derby.system.home ... Which should be advertised more aggressively. I've managed to get derby to listen to the host-add ( not localhost) I've managed to get to derby running on my development pc and get to it from the internet. So now we go the production environment ... I.e. windows exchange server. And no go ... Connection refused Even trying to stopnetworkserver is refused . Anybody tried this constellation and succeded using derby ? Or did I just calculate without host ? I could use a hint.-) Since I'm using the embedded driver at the moment... And that is no good in this situation, since the application is not designed to this setup.
Derby in windows
reason for problems is always some 40 cm away from the screen. The derby standard startNetworkserver.bat did not contain any use of %DERBY_CMD_LINE_ARGS% So they were registered and so on ... But never used. Only a wrong stop command did uncover this bug in the startup batch. So now I'll continue the path ... Hoping this was the last surprice.-) Erik
Re: Using NetworkServerControl to access in-memory (only) tables of Embedded Derby
Hello, Kristian:
Thanks for your response.
I think I found what the problem is... When my IDE (IntelliJ IDEA) hits a
breakpoint, it suspends all threads. I find it funny that in all these
years of programming and using IDEA, I've never realized it.
(possibly because I was too occupied debugging the code at the breakpoint
and not thinking about other threads at that moment). I've searched
through the options but couldn't find any setting
which will prevent other threads from being suspended. Will continue
looking.
I am now curious, what IDE do you use that it worked for you? And did you
need to enable any special settings to allow all other threads to continue
execution?
Regards,
Pavel.
Kristian Waagan kristian.waa...@sun.com
Sent by: kristian.waa...@sun.com
02/16/2010 03:32 AM
Please respond to
Derby Discussion derby-user@db.apache.org
To
Derby Discussion derby-user@db.apache.org
cc
Subject
Re: Using NetworkServerControl to access in-memory (only) tables of
Embedded Derby
On 16.02.10 03:02, Pavel Bortnovskiy wrote:
Bryan: thank you for your response.
I do see the thread you mention:
Thread Group [derby.daemons]{10}(Daemon)
Thread [derby.antiGC]{1}(Daemon)(WAITING)
Thread [derby.rawStoreDaemon]{5}(Daemon)(TIMED_WAITING)
Thread [derby.NetworkServerStarter]{5}(Daemon)(WAITING)
Thread [NetworkServerThread_2]{5}(Daemon)(RUNNABLE)
Thread [DRDAConnThread_3]{5}(Daemon)(WAITING)
However the behavior is still the same - if any other thread stops (for
instance in debugger), NetworkServerControl becomes unresponsive.
For instance, if I connect to it from the outside with another db app
(Aqua Data Studio, for instance), then connection to it breaks or the db
app can't access it.
But as soon as I let the thread run in that debugger, the connection
becomes alive and everything seems to work.
Has anyone else seen this kind of behavior?
Or perhaps, someone can share a piece of code (best practice) of how to
instantiate and run the NetworkServerControl, so that it's not
experiencing
such hang-ups.
Hi Pavel,
The code I'm about to post is by no means to be considered best
practice, but I think it does what you want it to do. However, the code
is basically yours :) (I copied it from your mail)
When I hit the breakpoint (set on the System.out.println in the static
main method), the ping thread continued to run, and I could also access
and do things with the database through ij.
Are you sure you're only stopping the current thread in your debugger?
When the system appears to hang, can you obtain a stack trace (using
jstack or similar) and check what the Derby threads are doing?
Are you sure your stopped thread isn't holding on to some monitors /
locks required by the other threads? (sounds unlikely, as the Derby
threads should operate on their own, but I'm asking anyway...)
See the code I tested with below (I'm using localhost as host, so you
cannot connect to Derby remotely).
Regards,
--
Kristian
import java.io.PrintWriter;
import java.net.InetAddress;
import java.util.concurrent.atomic.AtomicBoolean;
import org.apache.derby.drda.NetworkServerControl;
public class ServerTestDebug implements Runnable {
private final InetAddress m_host;
private final int m_port = 1527;
private final AtomicBoolean m_done = new AtomicBoolean(false);
private final int m_sleep = 5000;
private NetworkServerControl m_server;
public ServerTestDebug()
throws Exception {
m_host = InetAddress.getByName(localhost);
}
final public void start() throws Exception {
new Thread(this, Derby Server).start();
}
public void stop() {
System.out.println(Telling server to stop...);
m_done.set(true);
}
public void run() {
try {
m_server = new NetworkServerControl(m_host, m_port);
m_server.start(new PrintWriter(System.out));
} catch (Exception e) {
e.printStackTrace(System.out);
return;
}
try {
while (!m_done.get()) {
System.out.println(pinging...);
m_server.ping();
try {
Thread.sleep(m_sleep);
} catch (InterruptedException e) {
/* log */
}
}
} catch (Exception e) {
/* log - errror pinging server */
}
try {
System.out.println(Shutting down...);
m_server.shutdown();
} catch (Exception e) {
e.printStackTrace();
}
}
public static void main(String[] args)
throws Exception {
ServerTestDebug server = new ServerTestDebug();
server.start();
System.out.println(Starting main loop...);
int iterations = 30;
while (iterations 0) {
if (iterations == 20) {
System.out.println(breakpoint here!);
}
try
Re: Using NetworkServerControl to access in-memory (only) tables of Embedded Derby
Thank you, Kristian and Bryan for all your help. I did locate the setting
which allows me to suspend on the thread which hit the breakpoint while
allowing others to execute.
But it was quite an interesting exercise, nevertheless.
Pavel Bortnovskiy/JEFCO
02/18/2010 12:47 PM
To
Derby Discussion derby-user@db.apache.org
cc
kristian.waa...@sun.com
Subject
Re: Using NetworkServerControl to access in-memory (only) tables of
Embedded Derby
Hello, Kristian:
Thanks for your response.
I think I found what the problem is... When my IDE (IntelliJ IDEA) hits a
breakpoint, it suspends all threads. I find it funny that in all these
years of programming and using IDEA, I've never realized it.
(possibly because I was too occupied debugging the code at the breakpoint
and not thinking about other threads at that moment). I've searched
through the options but couldn't find any setting
which will prevent other threads from being suspended. Will continue
looking.
I am now curious, what IDE do you use that it worked for you? And did you
need to enable any special settings to allow all other threads to continue
execution?
Regards,
Pavel.
Kristian Waagan kristian.waa...@sun.com
Sent by: kristian.waa...@sun.com
02/16/2010 03:32 AM
Please respond to
Derby Discussion derby-user@db.apache.org
To
Derby Discussion derby-user@db.apache.org
cc
Subject
Re: Using NetworkServerControl to access in-memory (only) tables of
Embedded Derby
On 16.02.10 03:02, Pavel Bortnovskiy wrote:
Bryan: thank you for your response.
I do see the thread you mention:
Thread Group [derby.daemons]{10}(Daemon)
Thread [derby.antiGC]{1}(Daemon)(WAITING)
Thread [derby.rawStoreDaemon]{5}(Daemon)(TIMED_WAITING)
Thread [derby.NetworkServerStarter]{5}(Daemon)(WAITING)
Thread [NetworkServerThread_2]{5}(Daemon)(RUNNABLE)
Thread [DRDAConnThread_3]{5}(Daemon)(WAITING)
However the behavior is still the same - if any other thread stops (for
instance in debugger), NetworkServerControl becomes unresponsive.
For instance, if I connect to it from the outside with another db app
(Aqua Data Studio, for instance), then connection to it breaks or the db
app can't access it.
But as soon as I let the thread run in that debugger, the connection
becomes alive and everything seems to work.
Has anyone else seen this kind of behavior?
Or perhaps, someone can share a piece of code (best practice) of how to
instantiate and run the NetworkServerControl, so that it's not
experiencing
such hang-ups.
Hi Pavel,
The code I'm about to post is by no means to be considered best
practice, but I think it does what you want it to do. However, the code
is basically yours :) (I copied it from your mail)
When I hit the breakpoint (set on the System.out.println in the static
main method), the ping thread continued to run, and I could also access
and do things with the database through ij.
Are you sure you're only stopping the current thread in your debugger?
When the system appears to hang, can you obtain a stack trace (using
jstack or similar) and check what the Derby threads are doing?
Are you sure your stopped thread isn't holding on to some monitors /
locks required by the other threads? (sounds unlikely, as the Derby
threads should operate on their own, but I'm asking anyway...)
See the code I tested with below (I'm using localhost as host, so you
cannot connect to Derby remotely).
Regards,
--
Kristian
import java.io.PrintWriter;
import java.net.InetAddress;
import java.util.concurrent.atomic.AtomicBoolean;
import org.apache.derby.drda.NetworkServerControl;
public class ServerTestDebug implements Runnable {
private final InetAddress m_host;
private final int m_port = 1527;
private final AtomicBoolean m_done = new AtomicBoolean(false);
private final int m_sleep = 5000;
private NetworkServerControl m_server;
public ServerTestDebug()
throws Exception {
m_host = InetAddress.getByName(localhost);
}
final public void start() throws Exception {
new Thread(this, Derby Server).start();
}
public void stop() {
System.out.println(Telling server to stop...);
m_done.set(true);
}
public void run() {
try {
m_server = new NetworkServerControl(m_host, m_port);
m_server.start(new PrintWriter(System.out));
} catch (Exception e) {
e.printStackTrace(System.out);
return;
}
try {
while (!m_done.get()) {
System.out.println(pinging...);
m_server.ping();
try {
Thread.sleep(m_sleep);
} catch (InterruptedException e) {
/* log */
}
}
} catch (Exception e) {
/* log - errror pinging server */
}
try {
System.out.println(Shutting down...);
m_server.shutdown();
} catch
Re: Using NetworkServerControl to access in-memory (only) tables of Embedded Derby
On 18.02.10 18:47, Pavel Bortnovskiy wrote:
Hello, Kristian:
Thanks for your response.
I think I found what the problem is... When my IDE (IntelliJ IDEA) hits a
breakpoint, it suspends all threads. I find it funny that in all these
years of programming and using IDEA, I've never realized it.
(possibly because I was too occupied debugging the code at the breakpoint
and not thinking about other threads at that moment). I've searched
through the options but couldn't find any setting
which will prevent other threads from being suspended. Will continue
looking.
I am now curious, what IDE do you use that it worked for you? And did you
need to enable any special settings to allow all other threads to continue
execution?
Hi Pavel,
Glad to hear you figured out what caused your problems.
For the record, I'm using NetBeans.
I haven't tuned any nobs, stopping the current thread only is the
default I believe. You can stop all threads on a breakpoint if you want
to, and you can also make a different thread the current thread,
interrupt threads, and suspend threads.
Regards,
--
Kristian
Regards,
Pavel.
Kristian Waagankristian.waa...@sun.com
Sent by: kristian.waa...@sun.com
02/16/2010 03:32 AM
Please respond to
Derby Discussionderby-user@db.apache.org
To
Derby Discussionderby-user@db.apache.org
cc
Subject
Re: Using NetworkServerControl to access in-memory (only) tables of
Embedded Derby
On 16.02.10 03:02, Pavel Bortnovskiy wrote:
Bryan: thank you for your response.
I do see the thread you mention:
Thread Group [derby.daemons]{10}(Daemon)
Thread [derby.antiGC]{1}(Daemon)(WAITING)
Thread [derby.rawStoreDaemon]{5}(Daemon)(TIMED_WAITING)
Thread [derby.NetworkServerStarter]{5}(Daemon)(WAITING)
Thread [NetworkServerThread_2]{5}(Daemon)(RUNNABLE)
Thread [DRDAConnThread_3]{5}(Daemon)(WAITING)
However the behavior is still the same - if any other thread stops (for
instance in debugger), NetworkServerControl becomes unresponsive.
For instance, if I connect to it from the outside with another db app
(Aqua Data Studio, for instance), then connection to it breaks or the db
app can't access it.
But as soon as I let the thread run in that debugger, the connection
becomes alive and everything seems to work.
Has anyone else seen this kind of behavior?
Or perhaps, someone can share a piece of code (best practice) of how to
instantiate and run the NetworkServerControl, so that it's not
experiencing
such hang-ups.
Hi Pavel,
The code I'm about to post is by no means to be considered best
practice, but I think it does what you want it to do. However, the code
is basically yours :) (I copied it from your mail)
When I hit the breakpoint (set on the System.out.println in the static
main method), the ping thread continued to run, and I could also access
and do things with the database through ij.
Are you sure you're only stopping the current thread in your debugger?
When the system appears to hang, can you obtain a stack trace (using
jstack or similar) and check what the Derby threads are doing?
Are you sure your stopped thread isn't holding on to some monitors /
locks required by the other threads? (sounds unlikely, as the Derby
threads should operate on their own, but I'm asking anyway...)
See the code I tested with below (I'm using localhost as host, so you
cannot connect to Derby remotely).
Regards,
Re: Derby in windows
On 18.02.10 16:07, Erik Steen Jakobsen wrote: reason for problems is always some 40 cm away from the screen. The derby standard startNetworkserver.bat did not contain any use of %DERBY_CMD_LINE_ARGS% So they were registered and so on ... But never used. Only a wrong stop command did uncover this bug in the startup batch. Hi Erik, I found an existing issue [1] that fits your description of the problem, and I attached a patch to it. Thanks for the feedback, the problem will hopefully be fixed in the coming releases. Just for reference, which Derby version are you using? Regards, -- Kristian [1] https://issues.apache.org/jira/browse/DERBY-3507 So now I'll continue the path ... Hoping this was the last surprice.-) Erik
Re: Deadlocked in Log2File
We do not call SYSCS_UTIL.SYSCS_FREEZE_DATABASE or SYSCS_UTIL.SYSCS_CHECKPOINT_DATABASE. We do perform backups using CALL SYSCS_UTIL.SYSCS_BACKUP_DATABASE(?). However, in this particular test case the backup hasn't been run. So, for this particular test case the backup_database stored proc hasn't been called. This is a new application that uses derby so I can't say if this is new behavior for this application or not. We don't do any replication. This seems to happen fairly frequently if I try and process a very large data set for my application (70,000 entries). For smaller data set sizes it seems to work out ok (7500 entries). I encountered this from within my profiler so I decided to try this outside the profiler, and I didn't encounter any locking issue. If I do this within my profiler I see it frequently. For right now I think it's the profiler unless I can reproduce this outside my profiler. I'm using Derby 10.5.3. Thanks, Charlie On Wed, Feb 17, 2010 at 6:12 PM, Bryan Pendleton bpendle...@amberpoint.comwrote: I'm sometimes getting deadlocks in Log2File where all my worker threads are getting blocked, but I can't figure out which monitor they are blocked on. Thanks for sending the thread dump, it was very interesting. I agree with you, it's not clear what is blocking these threads, at least from the thread dump. I think this could be a Derby bug. How often does it happen? Is it a new behavior? Looking at the source code, LogToFile.flush() has some complicated synchronization logic which interacts with the backup/checkpoint methods. Does your application perform a backup? Does your application involve replication? Does your application ever call SYSCS_UTIL.SYSCS_FREEZE_DATABASE or SYSCS_UTIL.SYSCS_CHECKPOINT_DATABASE? If you are calling SYSCS_FREEZE_DATABASE, is it possible that you have some path through your system which fails to subsequently call SYSCS_UNFREEZE_DATABASE? thanks, bryan
Re: GenericPreparedStatements leak due to DependencyManager?
Actually I see this as well. I had over 690,000 of these laying around, and I am using Derby 10.5.3. I don't know if I have a test case that I can share, but within my program when I profile it I see a lot of these. They look like they are weakly referenced, but the GC doesn't want to give them up. Charlie On Wed, Feb 17, 2010 at 6:34 PM, Knut Anders Hatlen knut.hat...@sun.comwrote: Sumedh Wale sumw...@fastmail.fm writes: Hi I noticed OOM issues in one of the test runs that creates quite a good number of PreparedStatements. Looking at the hprof dump and the code, it looks like GenericPreparedStatements are being held on to by BasicDependencyManager even after having being expired from LCC's CacheManager. I looked at JIRA but did not find any existing bug for this -- is it a known issue? One fix can be to explicitly invoke DependencyManager#clearDependencies() for the GenericPreparedStatement from CachedStatement#clearIdentity(). This is with 10.4.x and I have not yet tried with latest 10.5 release. Hi Sumedh, I haven't heard about this issue before, so if you still see it with 10.5 it would be great if you could file a bug report with instructions on how to reproduce it. Thanks, -- Knut Anders
