Re: Job Listeners

[chandrika]

Hello Alexey,

could u please guide me with the above post . Thanks.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Job Listeners

[chandrika]

Hello Alexey,


on single node it is working fine till the job 8, from job7 there is no
execution as given in the below console logs: jobs getting executed are 12,
1, 11, 3, 2, 13, 9, 8..after that the jobs 7,4,5,6,10 are never getting
executed.




executed the job **  
12**15:59:11:012
incoming edgee**15:59:11:028
sourceVertex  incomingEdge is root**15:59:11:030
targetVertex incomingEdge is 1**15:59:11:035
15:59:11.038
[QuartzScheduler_DistributedNewScheduler123456-DESKTOP-DDN7SUT1507890505772_ClusterManager]
DEBUG org.quartz.impl.jdbcjobstore.JobStoreTX - ClusterManager: Check-in
complete.
outgoing edge(1 : 2) 15:59:11:048
sourceVertex is 1 15:59:11:049
targetVertex is 2 15:59:11:049
outgoing edge(1 : 3) 15:59:11:049
sourceVertex is 1 15:59:11:049
targetVertex is 3 15:59:11:049
executed the job **  
1**15:59:11:062
executed the job **  
11**15:59:11:091
executed the job **  
3**15:59:19:066
incoming edgee**15:59:19:079
sourceVertex  incomingEdge is root**15:59:19:080
targetVertex incomingEdge is 12**15:59:19:080
outgoing edge(12 : 13) 15:59:19:086
sourceVertex is 12 15:59:19:087
targetVertex is 13 15:59:19:087
executed the job **  
2**15:59:19:151
15:59:26.017
[QuartzScheduler_DistributedNewScheduler123456-DESKTOP-DDN7SUT1507890505772_MisfireHandler]
DEBUG org.quartz.impl.jdbcjobstore.JobStoreTX - MisfireHandler: scanning for
misfires...
15:59:26.032
[QuartzScheduler_DistributedNewScheduler123456-DESKTOP-DDN7SUT1507890505772_MisfireHandler]
DEBUG org.quartz.impl.jdbcjobstore.JobStoreTX - Found 0 triggers that missed
their scheduled fire-time.
15:59:26.046
[QuartzScheduler_DistributedNewScheduler123456-DESKTOP-DDN7SUT1507890505772_ClusterManager]
DEBUG org.quartz.impl.jdbcjobstore.JobStoreTX - ClusterManager: Check-in
complete.
executed the job **  
13**15:59:27:188
incoming edgee**15:59:27:201
sourceVertex  incomingEdge is root**15:59:27:202
targetVertex incomingEdge is 11**15:59:27:202
incoming edgee**15:59:27:219
sourceVertex  incomingEdge is 1**15:59:27:219
targetVertex incomingEdge is 2**15:59:27:219
outgoing edge(2 : 9) 15:59:27:222
sourceVertex is 2 15:59:27:223
targetVertex is 9 15:59:27:223
outgoing edge(2 : 8) 15:59:27:223
sourceVertex is 2 15:59:27:223
targetVertex is 8 15:59:27:224
outgoing edge(2 : 7) 15:59:27:224
sourceVertex is 2 15:59:27:224
targetVertex is 7 15:59:27:224
15:59:29.421 [DistributedNewScheduler123456_QuartzSchedulerThread] DEBUG
org.quartz.core.QuartzSchedulerThread - batch acquisition of 0 triggers
executed the job **  
9**15:59:35:253
incoming edgee**15:59:35:267
sourceVertex  incomingEdge is 1**15:59:35:267
targetVertex incomingEdge is 3**15:59:35:267
checkkk3
outgoing edge(3 : 6) 15:59:35:279
sourceVertex is 3 15:59:35:280
targetVertex is 6 15:59:35:280
outgoing edge(3 : 5) 15:59:35:282
sourceVertex is 3 15:59:35:282
targetVertex is 5 15:59:35:282
outgoing edge(3 : 4) 15:59:35:283
sourceVertex is 3 15:59:35:283
targetVertex is 4 15:59:35:283
executed the job **  
8**15:59:35:324
15:59:41.051
[QuartzScheduler_DistributedNewScheduler123456-DESKTOP-DDN7SUT1507890505772_ClusterManager]
DEBUG org.quartz.impl.jdbcjobstore.JobStoreTX - ClusterManager: Check-in
complete.
15:59:52.866 [DistributedNewScheduler123456_QuartzSchedulerThread] DEBUG
org.quartz.core.QuartzSchedulerThread - batch acquisition of 0 triggers




thanks,
chandrika



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Job Listeners

[chandrika]

Hello Alexey,

Even i could make my code work on three nodes even earlier, but with one
node it was always failing causing a deadlock, please let me know how to go
about it cause the issue was with one node.

thanks 
chandrika



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Job Listeners

[chandrika]

Hello Alexey,


the sample code is as given below:

@ComputeTaskSessionFullSupport
public class SplitExampleJgraphWithComplexDAGIgniteCachesample extends
ComputeTaskSplitAdapter ,
Integer> {

   
// Auto-injected task session.
@TaskSessionResource 
private ComputeTaskSession ses;


private static final Random random = new Random();
static int noOftasksExecutedSuccess = 0;

SimpleDateFormat dateFormat = new SimpleDateFormat("HH:mm:ss:SSS");




@Override protected Collection split(int
clusterSize, CustomDirectedAcyclicGraph graph) {
Collection jobs = new LinkedList<>();

IgniteCache cacheUp = 
Ignition.ignite().getOrCreateCache("cacheNameNew");
ses.addAttributeListener((key, val) -> {
if ("COMPLETE".compareTo(key.toString()) == 0) {
nextTaskToExecute(graph, cacheUp);
}
}, false);

String task = null;
if (cacheUp.get("CurrentVertex") != null)
task = (String) cacheUp.get("CurrentVertex");
for (DefaultEdge outgoingEdge : graph.outgoingEdgesOf(task)) {
String sourceVertex = graph.getEdgeSource(outgoingEdge);
String targetVertex = graph.getEdgeTarget(outgoingEdge);
graph.setTargetVertex(targetVertex);
executingJobsBuilt(graph, jobs);
} if (task != null && graph.outgoingEdgesOf(task).size() == 0) {
if (cacheUp.get(task) != null && (Boolean)cacheUp.get(task)) {
String targetVertex = setNextVertexInCache(graph, 
cacheUp);
graph.setTargetVertex(targetVertex);
nextTaskToExecute(graph, cacheUp);
} else {
System.out.println("else part");
}
}
return jobs;
}



private void nextTaskToExecute(CustomDirectedAcyclicGraph graph,
IgniteCache cacheUp) {
Ignite ignite = Ignition.ignite();
if (cacheUp.get("NextVertex") != null) {
String processingVertex = (String) 
cacheUp.get("NextVertex");
if (processingVertex != null && 
areParentVerticesProcessed(graph,
processingVertex, cacheUp)) {
cacheUp.put("CurrentVertex", processingVertex);
// Execute task on the cluster and wait for its 
completion.

ignite.compute().execute(SplitExampleJgraphWithComplexDAGIgniteCachesample.class,
graph);
}
}
}

private void executingJobsBuilt(CustomDirectedAcyclicGraph graph, Collection jobs) {
String targetVertex = graph.getTargetVertex();
IgniteCache cacheNew = 
Ignition.ignite().getOrCreateCache("cacheNameNew");
   if (targetVertex != null && !cacheNew.containsKey(targetVertex)) {
   jobs.add(new ComputeJobAdapter() {
   // Auto-injected job context.
@JobContextResource
private ComputeJobContext jobCtx;

   @Nullable @Override public Object execute() {
   int duration1 = 8000 + random.nextInt(100);
   SimpleDateFormat dateFormatNew = new
SimpleDateFormat("HH:mm:ss:SSS");
   String task = (String) targetVertex;
   try {
   Thread.sleep(duration1);
   
System.out.println("executed the job **  
" + task +  "**" + dateFormatNew.format(new Date()));
cacheNew.put(task, true);
   } catch (Exception e1) {
e1.printStackTrace();
} 
ses.setAttribute("NEXTVERTEX", 
setNextVertexInCache(graph, cacheNew));
ses.setAttribute("COMPLETE", duration1);
   return duration1;
   }
   
   });
   
   }
   
   }


private String setNextVertexInCache(CustomDirectedAcyclicGraph graph, IgniteCache cache) {
String task = null; 
Set dagSourceVertex = graph.vertexSet();
Iterator itr = dagSourceVertex.iterator();
  while (itr.hasNext()) {
task = (String)itr.next();
if(cache.get("CurrentVertex") != null &&
!task.equalsIgnoreCase((String)cache.get("CurrentVertex")))
continue;
else {

Re: Job Listeners

[chandrika]

Hello Alexey,

Thanks for the valuable information, i hare tried executing a list of
dependent tasks using a DAG using session.setAttribute("COMPLETE",true), and
it is working fine as long as there are three nodes, cause there are 3 or
less parallel tasks to execute.

But when i run the same code of dependent tasks on a single node, then it
stops its execution after execution of few tasks in some time . There are no
exception as such. could u please let me know where i could have gone wrong.
Note : should i have to use waitForAttribute for the same if so could u
please guide me the usage of wiatForAttribute for completion of three
parallel tasks at every step.


thanks and regards,
chandrika



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Job Listeners

[chandrika]

Hello Alexey,

thanks a lot for the input it was very useful, there are two more things i m
stuck at:

1. when i run the above in cluster environment(with more than one node) with
value as an object in session.setAttribute(key, value) value being an
object, then i m unable to proceed further as one of the jobs is getting
rejected on another slave node. the object in the value is serializable.
could u please guide me how to track down what could be the reason for job
to be rejected.

2. also on another note , if i wanted to get the detailed information of the
job as given below
JVM start time   | 03/14/16, 10:53:49   | 
| Node start time  | 03/14/16, 10:53:58   | 
| Up time  | 00:21:31:692 | 
| Last metric update   | 03/14/16, 11:15:20   | 
| CPUs | 4| 
| Thread count | 91   | 
| Cur/avg active jobs  | 0/0.01   | 
| Cur/avg waiting jobs | 0/0.00   | 
| Cur/avg rejected jobs| 0/0.00   | 
| Cur/avg cancelled jobs   | 0/0.00   | 
| Cur/avg job wait time| 0/0.00ms | 
| Cur/avg job execute time | 0/11486.00ms | 
| Cur/avg CPU load %   | 0.13/0.39%

is there a mechanism other than the IgnitePredicate.

thanks and regards,
chandrika



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Job Listeners

[chandrika]

Hello Alexey,

Thanks a lot for the information it was pretty useful to us,also wanted to
know if we could know the percentage of the JobSibling Completed or task
Completed, as in 70% of it is finished its execution and 30% more is left to
finish the execution or the duration of time taken for executing the job.
Please guide us through in this aspect.

thanks and regards,
chandrika




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Job Listeners

[chandrika]

Hello All,

we have a parent task which is comprised of three jobsiblings, we need to
handle dependent tasks.
So i was looking for listeners for jobsiblings and parent tasks to handle
dependencies. please let me know if there are any event listeners of
JobWasExecuted, JobToBeExecuted for JobSiblings as well as for Parent Tasks
in ignite.

thanks and regards,
Chandrika



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

ignite cron scheduling in cluster

[chandrika]

Hello All,

Apache ignite was very useful in our product, but have a query regarding
schedulers cron as given below:

We noticed that we can provide cron expression for a job to run only on
local node (i mean with the cron the job will run recurrently in the local
node only). Is there anyway for us to configure a job with cron expression
and schedule it on a cluster as and when the time reached to run the job, it
needs to be executed on one of the available nodes. 

Also in case of node failure in cluster on which  the job has started its
execution, do we have a mechanism to restore . When an exception in Jobs
there are check points to restore but when the node on whcih the execution
of the job has started, if that fails hoe do we restore back the job on
another runnning node.

please get back to us on the above points as we are stuck in this aspect.

Thanks and Regards,
Chandrika



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/ignite-cron-scheduling-in-cluster-tp16513.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: fetching all the tasks already scheduled and to know the status of the task

[chandrika]

Hello Alex,

Thanks a lot for a quick reply .

Have one more query regarding the above point 2 mentioned in the earlier
post , we have a way to get using ComputeTaskFuture is what was said but if
i have to use SchedulerFuture  for getting the below information , then how
do we go about it.
1. fetching all the tasks in the ignite nodes
2. all the jobs associated with a task

our task is defined is as given below:
SchedulerFuture fut = ignite.scheduler().scheduleLocal(..)

the reason why i am asking cause i need to schedule all the jobs with a cron
expression which is our requirement.

thanks and regards,
chandrika



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/fetching-all-the-tasks-already-scheduled-and-to-know-the-status-of-the-task-tp16393p16443.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Cron expression in agnite-schedulers

[chandrika]

Hello All,

so far we had a good experience with ignite and we are using
ignite-scheduler in our project.
Cron expression is used for scheduling the task as given below:

SchedulerFuture fut = ignite.scheduler().scheduleLocal(
new Callable() {
private int invocations;

@Override public Integer call() {
invocations++;
ignite.compute().broadcast(
new IgniteRunnable() {
@Override public void run() {
  //TODO task to be executed is coded
here
 ...
 ...
  }
}
);

return invocations;
}
},
   "*/1 * * * *");

With the above cron expression it is working fine but when wanted to give
exact date and time as in * */1 18 27 8 ? 2017 (27th august , 2017 at 6:00
pm) then we are unable to schedule using apache-ignite-scheduler. The above
cron is a valid cron expression.
please let me know whether i am going in the right direction or not. If not
could you please guide me through.

Thanks and regards,
chandrika





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cron-expression-in-agnite-schedulers-tp16433.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.