Re: [PR] [Feature][Engine] Change the API to concurrent query [seatunnel]

[via GitHub]

KAILINYmq commented on PR #9733:
URL: https://github.com/apache/seatunnel/pull/9733#issuecomment-3223252865

   > Did you some config change? I can't reproduce it. <img alt="image" 
width="1256" height="628" 
src="https://private-user-images.githubusercontent.com/32387433/481948456-17b55253-ebca-4b43-9949-c3c27d3971de.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTYxOTE5MzcsIm5iZiI6MTc1NjE5MTYzNywicGF0aCI6Ii8zMjM4NzQzMy80ODE5NDg0NTYtMTdiNTUyNTMtZWJjYS00YjQzLTk5NDktYzNjMjdkMzk3MWRlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA4MjYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwODI2VDA3MDAzN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTQwMTE5Y2RlNjJhMDhkN2Q5NDUwNWJhYWJjODk5YzIyYTM0MDRmNTZkODQwMGQyYTExNDEzZjA2NmQ1YjQxOGImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.eQ4gPyl9J233WTDCE7GR69XRNXm3Y-3y0VSOLxzvfi0";>
 <img alt="image" width="1528" height="590" 
src="https://private-user-images.githubusercontent.com/32387433/4819485
 
12-9cd339fd-597a-4914-b31d-c9703b74b4cf.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTYxOTE5MzcsIm5iZiI6MTc1NjE5MTYzNywicGF0aCI6Ii8zMjM4NzQzMy80ODE5NDg1MTItOWNkMzM5ZmQtNTk3YS00OTE0LWIzMWQtYzk3MDNiNzRiNGNmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA4MjYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwODI2VDA3MDAzN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTU3ZjhkZTI2MDZiMGY5YzQ1OGIyYTllNDZhOWFmMmQyNDNjYjMxYzkzNjA2YjhkZDBkM2JiZTQ3ZTI2MDAxN2YmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.uVQUjBksvBNXoP-HMDgcPsT5p26z096ZYL2tXqk-W_Q">
   
   
   
   My longer time may be due to configuring OSS, while other configurations are 
default. But these are only 12 tasks, in the Pro environment, hundreds of tasks 
would take a long time to complete.
   <img width="2289" height="850" alt="image" 
src="https://github.com/user-attachments/assets/698df70b-0752-4938-ae10-9ea72116bc27";
 />
   
   Here is my local configuration，All other configurations are default：
   seatunnel.yaml
   ```
   seatunnel:
     engine:
       classloader-cache-mode: true
       history-job-expire-minutes: 1440
       backup-count: 1
       queue-type: blockingqueue
       print-execution-info-interval: 60
       print-job-metrics-info-interval: 60
       slot-service:
         dynamic-slot: true
       checkpoint:
         interval: 10000
         timeout: 60000
         storage:
           type: hdfs
           max-retained: 3
           plugin-config:
             namespace: /tmp/seatunnel/checkpoint_snapshot
             storage.type: hdfs
             fs.defaultFS: file:///tmp/ # Ensure that the directory has written 
permission
       telemetry:
         metric:
           enabled: false
         logs:
           scheduled-deletion-enable: true
       http:
         enable-http: true
         port: 8080
         enable-dynamic-port: false
   ```
   hazelcast-master.yaml
   ```
   hazelcast:
     cluster-name: seatunnel
     network:
       rest-api:
         enabled: false
         endpoint-groups:
           CLUSTER_WRITE:
             enabled: true
           DATA:
             enabled: true
       join:
         tcp-ip:
           enabled: true
           member-list:
             - localhost:5801
             - localhost:5802
       port:
         auto-increment: false
         port: 5801
     properties:
       hazelcast.invocation.max.retry.count: 20
       hazelcast.tcp.join.port.try.count: 30
       hazelcast.logging.type: log4j2
       hazelcast.operation.generic.thread.count: 50
       hazelcast.heartbeat.failuredetector.type: phi-accrual
       hazelcast.heartbeat.interval.seconds: 2
       hazelcast.max.no.heartbeat.seconds: 180
       hazelcast.heartbeat.phiaccrual.failuredetector.threshold: 10
       hazelcast.heartbeat.phiaccrual.failuredetector.sample.size: 200
       hazelcast.heartbeat.phiaccrual.failuredetector.min.std.dev.millis: 100
     map:
       default:
         backup-count: 2
       engine*:
         map-store:
           enabled: true
           initial-mode: EAGER
           factory-class-name: 
org.apache.seatunnel.engine.server.persistence.FileMapStoreFactory
           properties:
             type: hdfs
             namespace: /local_seatunnel/imap
             clusterName: seatunnel
             storage.type: oss
             block.size: 1073741824
             oss.bucket: oss://base/seatunnel/
             fs.oss.accessKeyId: ********
             fs.oss.accessKeySecret: ********
             fs.oss.endpoint: ********
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org