Re: Performance Questions - [SOLVED (apparently)]
I thought I should update the list, following our testing a clone of the database, with a timer to delay between the devices without actually sending. It appears the combination of turning concurrency on per Pascal (seems to me that should be the default) and then using Kieran's suggestions for extending ERXAbstractTask and using a newEditingContext to spawn a pool of OSCs were each dramatic improvements. Now concurrency no longer appears to be an issue. Thanks everyone! Andrew On Jul 31, 2011, at 5:29 PM, Kieran Kelleher wrote: > IMMEDIATE BAND-AID 60 second FIX > > 1. Go to Monitor, create a new 'Application' named "Admin" and select the > current application executable path, launch an instance and use that Admin > instance for sending notifications. (You don't need to copy the app, rename > and redeploy to do this as you suggested!) > 2. Restart your "Public" instance with properties in WOMonitor to (1) provide > multiple OSC's for regular requests using wonder property for the ERXOSC > pool, and (2) to turn on concurrency. > 3. If your public instance(s) struggle(s) on the next batch of notifications, > add more instances. > > > > MEDIUM, LONG TERM IMPROVEMENT > === > > You only have 76 MB of data, do you expect this to grow significantly in the > next year or two? > > Your innodb memory allocation is 128MB, so your entire DB fits in memory, > albeit there are a few settings that can be tweaked even as it is right now. > > For high concurrency on this small dataset, you are better off with a few > CPUs and (if necessary to offset costs, less memory than 16GB). Not sure if > your server is a physical machine or a VPS that can be reconfigured. > > Before I would make recomendations on config settings, you need to decide how > much memory you want MySQL to use based on projected peak data size over next > 12 months 256MB, 512MB, 1GB, 2GB ??? > > From what I understand, your current issues are related to high concurrency > in short periods during and after the times when you send your 20,000 > notifications, and IIRC, you have database/woa and apache on one machine with > 16GB RAM and 1 CPU > > Here is some recommendations to handle that concurrency better. You can do > the first 3 right now after the BAND-AID fix above gives you breathing space: > * Turn on concurrency in your app. > * Use ERXObjectStoreCoordinatorPool for handling regular request-response > EOF. Start with 3 OSC's per instance and see how it goes. > * Use a background thread for your NotificationsSending process with its own > OSC (easy way: just extend ERXAbstractTask, use its newEditingContext() > method in your task, or copy the logic in that class to your Runnable > background task.) > * If your server is a VPS that can have its configuration changed, consider > more CPUs (4?) and less total memory, if that offsets the cost of more CPUs, > with memory based on next 12 month peak traffic and data size expectations > (4GB?) > > > Then > * Monitor memory for your app instance(s) and adjust as needed. > * You might find that you handle the peak traffic bursts by having a few > instances rather than just one > * Monitor, measure and adjust again. > > So just maybe an ideal config for your scenario right now might be sth like > this: > > MySQL: allocate 512MB, covers 6x data size growth, and edit my.cnf to make > the best use of that 512MB. > WOA App Instances: 4-6 instances x ???MB each > > > Some other observations: > TABLE INDICES > --- > * Every index adds time to inserts, updates and adds space to the database > size, so no need for redundant indices. For example, the following join table > has 3 indices and all (single and compound indices) begin with 'app_dev_Id', > so therefore this one, KEY `app_dev_Id` (`app_dev_Id`), is redundant. Either > of the other two covers that index requirement. > > > CREATE TABLE IF NOT EXISTS `app_dev_not_type` ( > `app_dev_Id` bigint(20) NOT NULL default '0', > `not_type_id` int(11) NOT NULL default '0', > `active` int(11) default '1', > `create_date` datetime default NULL, > `modify_date` datetime default NULL, > PRIMARY KEY (`app_dev_Id`,`not_type_id`), > KEY `app_dev_Id` (`app_dev_Id`), > KEY `app_dev_Id_3` (`app_dev_Id`,`active`), > KEY `idx_not_type_id` (`not_type_id`) > ) ENGINE=InnoDB DEFAULT CHARSET=latin1; > > is the same as: > > CREATE TABLE IF NOT EXISTS `app_dev_not_type` ( > `app_dev_Id` bigint(20) NOT NULL default '0', > `not_type_id` int(11) NOT NULL default '0', > `active` int(11) default '1', > `create_date` datetime default NULL, > `modify_date` datetime default NULL, > PRIMARY KEY (`app_dev_Id`,`not_type_id`), > KEY `app_dev_Id_3` (`app_dev_Id`,`active`), > KEY `idx_not_type_id` (`not_type_id`) > ) ENGINE=InnoDB DEFAULT CHARSET=latin1; > > > * it is a good practice to add index for the reverse relationship in join > tables, for examp
Re: Performance Questions
Thanks. We already implemented several of these (based on Pascal's suggestion about concurrency being turned on, I assumed that was the default but was apparently wrong). Concurrency and 4 instances have dramatically reduced the occurrences of long wait times. Down to something like 4 out of many thousands of accesses. I will be doing the separate thread/concurrency pool thing first thing in the AM. Thanks a bunch! I would not have even known where to look without this list. Sent from my iPad On Jul 31, 2011, at 5:29 PM, Kieran Kelleher wrote: > IMMEDIATE BAND-AID 60 second FIX > > 1. Go to Monitor, create a new 'Application' named "Admin" and select the > current application executable path, launch an instance and use that Admin > instance for sending notifications. (You don't need to copy the app, rename > and redeploy to do this as you suggested!) > 2. Restart your "Public" instance with properties in WOMonitor to (1) provide > multiple OSC's for regular requests using wonder property for the ERXOSC > pool, and (2) to turn on concurrency. > 3. If your public instance(s) struggle(s) on the next batch of notifications, > add more instances. > > > > MEDIUM, LONG TERM IMPROVEMENT > === > > You only have 76 MB of data, do you expect this to grow significantly in the > next year or two? > > Your innodb memory allocation is 128MB, so your entire DB fits in memory, > albeit there are a few settings that can be tweaked even as it is right now. > > For high concurrency on this small dataset, you are better off with a few > CPUs and (if necessary to offset costs, less memory than 16GB). Not sure if > your server is a physical machine or a VPS that can be reconfigured. > > Before I would make recomendations on config settings, you need to decide how > much memory you want MySQL to use based on projected peak data size over next > 12 months 256MB, 512MB, 1GB, 2GB ??? > > From what I understand, your current issues are related to high concurrency > in short periods during and after the times when you send your 20,000 > notifications, and IIRC, you have database/woa and apache on one machine with > 16GB RAM and 1 CPU > > Here is some recommendations to handle that concurrency better. You can do > the first 3 right now after the BAND-AID fix above gives you breathing space: > * Turn on concurrency in your app. > * Use ERXObjectStoreCoordinatorPool for handling regular request-response > EOF. Start with 3 OSC's per instance and see how it goes. > * Use a background thread for your NotificationsSending process with its own > OSC (easy way: just extend ERXAbstractTask, use its newEditingContext() > method in your task, or copy the logic in that class to your Runnable > background task.) > * If your server is a VPS that can have its configuration changed, consider > more CPUs (4?) and less total memory, if that offsets the cost of more CPUs, > with memory based on next 12 month peak traffic and data size expectations > (4GB?) > > > Then > * Monitor memory for your app instance(s) and adjust as needed. > * You might find that you handle the peak traffic bursts by having a few > instances rather than just one > * Monitor, measure and adjust again. > > So just maybe an ideal config for your scenario right now might be sth like > this: > > MySQL: allocate 512MB, covers 6x data size growth, and edit my.cnf to make > the best use of that 512MB. > WOA App Instances: 4-6 instances x ???MB each > > > Some other observations: > TABLE INDICES > --- > * Every index adds time to inserts, updates and adds space to the database > size, so no need for redundant indices. For example, the following join table > has 3 indices and all (single and compound indices) begin with 'app_dev_Id', > so therefore this one, KEY `app_dev_Id` (`app_dev_Id`), is redundant. Either > of the other two covers that index requirement. > > > CREATE TABLE IF NOT EXISTS `app_dev_not_type` ( > `app_dev_Id` bigint(20) NOT NULL default '0', > `not_type_id` int(11) NOT NULL default '0', > `active` int(11) default '1', > `create_date` datetime default NULL, > `modify_date` datetime default NULL, > PRIMARY KEY (`app_dev_Id`,`not_type_id`), > KEY `app_dev_Id` (`app_dev_Id`), > KEY `app_dev_Id_3` (`app_dev_Id`,`active`), > KEY `idx_not_type_id` (`not_type_id`) > ) ENGINE=InnoDB DEFAULT CHARSET=latin1; > >is the same as: > > CREATE TABLE IF NOT EXISTS `app_dev_not_type` ( > `app_dev_Id` bigint(20) NOT NULL default '0', > `not_type_id` int(11) NOT NULL default '0', > `active` int(11) default '1', > `create_date` datetime default NULL, > `modify_date` datetime default NULL, > PRIMARY KEY (`app_dev_Id`,`not_type_id`), > KEY `app_dev_Id_3` (`app_dev_Id`,`active`), > KEY `idx_not_type_id` (`not_type_id`) > ) ENGINE=InnoDB DEFAULT CHARSET=latin1; > > > * it is a good practice to add index for the reverse relationship in join > t
Re: Performance Questions
IMMEDIATE BAND-AID 60 second FIX 1. Go to Monitor, create a new 'Application' named "Admin" and select the current application executable path, launch an instance and use that Admin instance for sending notifications. (You don't need to copy the app, rename and redeploy to do this as you suggested!) 2. Restart your "Public" instance with properties in WOMonitor to (1) provide multiple OSC's for regular requests using wonder property for the ERXOSC pool, and (2) to turn on concurrency. 3. If your public instance(s) struggle(s) on the next batch of notifications, add more instances. MEDIUM, LONG TERM IMPROVEMENT === You only have 76 MB of data, do you expect this to grow significantly in the next year or two? Your innodb memory allocation is 128MB, so your entire DB fits in memory, albeit there are a few settings that can be tweaked even as it is right now. For high concurrency on this small dataset, you are better off with a few CPUs and (if necessary to offset costs, less memory than 16GB). Not sure if your server is a physical machine or a VPS that can be reconfigured. Before I would make recomendations on config settings, you need to decide how much memory you want MySQL to use based on projected peak data size over next 12 months 256MB, 512MB, 1GB, 2GB ??? From what I understand, your current issues are related to high concurrency in short periods during and after the times when you send your 20,000 notifications, and IIRC, you have database/woa and apache on one machine with 16GB RAM and 1 CPU Here is some recommendations to handle that concurrency better. You can do the first 3 right now after the BAND-AID fix above gives you breathing space: * Turn on concurrency in your app. * Use ERXObjectStoreCoordinatorPool for handling regular request-response EOF. Start with 3 OSC's per instance and see how it goes. * Use a background thread for your NotificationsSending process with its own OSC (easy way: just extend ERXAbstractTask, use its newEditingContext() method in your task, or copy the logic in that class to your Runnable background task.) * If your server is a VPS that can have its configuration changed, consider more CPUs (4?) and less total memory, if that offsets the cost of more CPUs, with memory based on next 12 month peak traffic and data size expectations (4GB?) Then * Monitor memory for your app instance(s) and adjust as needed. * You might find that you handle the peak traffic bursts by having a few instances rather than just one * Monitor, measure and adjust again. So just maybe an ideal config for your scenario right now might be sth like this: MySQL: allocate 512MB, covers 6x data size growth, and edit my.cnf to make the best use of that 512MB. WOA App Instances: 4-6 instances x ???MB each Some other observations: TABLE INDICES --- * Every index adds time to inserts, updates and adds space to the database size, so no need for redundant indices. For example, the following join table has 3 indices and all (single and compound indices) begin with 'app_dev_Id', so therefore this one, KEY `app_dev_Id` (`app_dev_Id`), is redundant. Either of the other two covers that index requirement. CREATE TABLE IF NOT EXISTS `app_dev_not_type` ( `app_dev_Id` bigint(20) NOT NULL default '0', `not_type_id` int(11) NOT NULL default '0', `active` int(11) default '1', `create_date` datetime default NULL, `modify_date` datetime default NULL, PRIMARY KEY (`app_dev_Id`,`not_type_id`), KEY `app_dev_Id` (`app_dev_Id`), KEY `app_dev_Id_3` (`app_dev_Id`,`active`), KEY `idx_not_type_id` (`not_type_id`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1; is the same as: CREATE TABLE IF NOT EXISTS `app_dev_not_type` ( `app_dev_Id` bigint(20) NOT NULL default '0', `not_type_id` int(11) NOT NULL default '0', `active` int(11) default '1', `create_date` datetime default NULL, `modify_date` datetime default NULL, PRIMARY KEY (`app_dev_Id`,`not_type_id`), KEY `app_dev_Id_3` (`app_dev_Id`,`active`), KEY `idx_not_type_id` (`not_type_id`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1; * it is a good practice to add index for the reverse relationship in join tables, for example: CREATE TABLE IF NOT EXISTS `device_notification` ( `device_id` bigint(20) NOT NULL default '0', `notification_id` bigint(20) NOT NULL default '0', PRIMARY KEY (`device_id`,`notification_id`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1; You should put compound index for the reverse relationship, so it should be: CREATE TABLE IF NOT EXISTS `device_notification` ( `device_id` bigint(20) NOT NULL default '0', `notification_id` bigint(20) NOT NULL default '0', PRIMARY KEY (`device_id`,`notification_id`), KEY `reverse_rel` (`notification_id`,`device_id`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1; SUMMARY == Andrew, it seems your primary issue here is EOF concurrency and app concurrency in general during
Re: Performance Questions
OK, thanks. I thought I looked through the example, presumably I missed something. In any event, yes, watching the session again makes lots of sense. Maybe it will give me something to do while I avoid 100 degree heat this weekend. Andrew On Jul 29, 2011, at 4:56 PM, Kieran Kelleher wrote: > Check out wonder source and run the BackgroundTasks example and look a the > examples themselves. Also watch the video for that from this year's WOWODC. > Pascal sent it out to attendees recently. > > On Jul 29, 2011, at 4:53 PM, Andrew Kinnie wrote: > >> OK, I have done a little searching and poking around in source, and am not >> finding how one should do the multiple OSCs solution. >> >> I see there is a property to set to determine the size of the pool, and I >> see how to get an OSC from a pool, but I don't see anything else about what >> I need to do to take objects from one stack and put them into a new OSC and >> how to handle the ec, etc. Presumably I just need other search terms, but >> if anyone has an example I can look at that does this sort of thing, I would >> appreciate it. I want the task to run in it's own OSC it gets from the pool >> so that it does not slow down the other concurrent connections (concurrency >> is now on. Though I assumed it always was, frankly.) >> >> Andrew >> >> On Jul 29, 2011, at 2:56 PM, Kieran Kelleher wrote: >> >>> 2) For your 1 minute task do it in a background thread and use a different >>> OSC. Remember EOF is a single-threaded, single-db-connection stack. If you >>> want high concurrency performance, you cannot just use the default OSC. Use >>> a ERXTaskObjectStoreCoordinatorPool just for tasks, even if it is just a >>> pool of one. >> >> ___ >> Do not post admin requests to the list. They will be ignored. >> Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) >> Help/Unsubscribe/Update your Subscription: >> http://lists.apple.com/mailman/options/webobjects-dev/kelleherk%40gmail.com >> >> This email sent to kelleh...@gmail.com > ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Performance Questions
Check out wonder source and run the BackgroundTasks example and look a the examples themselves. Also watch the video for that from this year's WOWODC. Pascal sent it out to attendees recently. On Jul 29, 2011, at 4:53 PM, Andrew Kinnie wrote: > OK, I have done a little searching and poking around in source, and am not > finding how one should do the multiple OSCs solution. > > I see there is a property to set to determine the size of the pool, and I see > how to get an OSC from a pool, but I don't see anything else about what I > need to do to take objects from one stack and put them into a new OSC and how > to handle the ec, etc. Presumably I just need other search terms, but if > anyone has an example I can look at that does this sort of thing, I would > appreciate it. I want the task to run in it's own OSC it gets from the pool > so that it does not slow down the other concurrent connections (concurrency > is now on. Though I assumed it always was, frankly.) > > Andrew > > On Jul 29, 2011, at 2:56 PM, Kieran Kelleher wrote: > >> 2) For your 1 minute task do it in a background thread and use a different >> OSC. Remember EOF is a single-threaded, single-db-connection stack. If you >> want high concurrency performance, you cannot just use the default OSC. Use >> a ERXTaskObjectStoreCoordinatorPool just for tasks, even if it is just a >> pool of one. > > ___ > Do not post admin requests to the list. They will be ignored. > Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/webobjects-dev/kelleherk%40gmail.com > > This email sent to kelleh...@gmail.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Performance Questions
OK, I have done a little searching and poking around in source, and am not finding how one should do the multiple OSCs solution. I see there is a property to set to determine the size of the pool, and I see how to get an OSC from a pool, but I don't see anything else about what I need to do to take objects from one stack and put them into a new OSC and how to handle the ec, etc. Presumably I just need other search terms, but if anyone has an example I can look at that does this sort of thing, I would appreciate it. I want the task to run in it's own OSC it gets from the pool so that it does not slow down the other concurrent connections (concurrency is now on. Though I assumed it always was, frankly.) Andrew On Jul 29, 2011, at 2:56 PM, Kieran Kelleher wrote: > 2) For your 1 minute task do it in a background thread and use a different > OSC. Remember EOF is a single-threaded, single-db-connection stack. If you > want high concurrency performance, you cannot just use the default OSC. Use a > ERXTaskObjectStoreCoordinatorPool just for tasks, even if it is just a pool > of one. ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Performance Questions
OK, thanks. I have implemented a simple task version of my old utilities class to run the sends, but I did not know that about EOF being single threaded or the single db connection. I will look at the ERXTaskObjectStoreCoordinatorPool. BIG help. I attached the query results and the show variables result "TABLE_SCHEMA","TABLE_NAME","TABLE_ROWS","SIZE_IN_MB","DATA_SIZE_IN_MB","INDEX_SIZE_IN_MB" "apns","app_dev_not_type",195693,29.98437500,13.54687500,16.4375 "apns","device_notification",339498,24.45312500,24.45312500,0. "apns","application_device",50555,21.57812500,10.51562500,11.0625 "apns","notification",27,0.04687500,0.01562500,0.03125000 "apns","person",1,0.03125000,0.01562500,0.01562500 "apns","eo_pk_table",4,0.03125000,0.01562500,0.01562500 "apns","push_application",1,0.03125000,0.01562500,0.01562500 "apns","notification_type",5,0.03125000,0.01562500,0.01562500 "apns","_dbupdater",1,0.01562500,0.01562500,0. "apns","user_type",2,0.01562500,0.01562500,0. "information_schema","VIEWS",NULL,0.00097656,0.,0.00097656 "information_schema","TRIGGERS",NULL,0.00097656,0.,0.00097656 "information_schema","COLUMNS",NULL,0.00097656,0.,0.00097656 "information_schema","ROUTINES",NULL,0.00097656,0.,0.00097656 "information_schema","COLUMN_PRIVILEGES",NULL,0.,0.,0. "information_schema","SCHEMATA",NULL,0.,0.,0. "information_schema","TABLE_CONSTRAINTS",NULL,0.,0.,0. "information_schema","CHARACTER_SETS",NULL,0.,0.,0. "information_schema","KEY_COLUMN_USAGE",NULL,0.,0.,0. "information_schema","SCHEMA_PRIVILEGES",NULL,0.,0.,0. "information_schema","TABLE_PRIVILEGES",NULL,0.,0.,0. "information_schema","COLLATIONS",NULL,0.,0.,0. "information_schema","PROFILING",NULL,0.,0.,0. "information_schema","STATISTICS",NULL,0.,0.,0. "information_schema","COLLATION_CHARACTER_SET_APPLICABILITY",NULL,0.,0.,0. "information_schema","TABLES",NULL,0.,0.,0. "information_schema","USER_PRIVILEGES",NULL,0.,0.,0. "Variable_name","Value" "auto_increment_increment",1 "auto_increment_offset",1 "automatic_sp_privileges","ON" "back_log",50 "basedir","/usr/" "bdb_cache_size",8384512 "bdb_home","/var/lib/mysql/" "bdb_log_buffer_size",262144 "bdb_logdir","" "bdb_max_lock",1 "bdb_shared_data","OFF" "bdb_tmpdir","/tmp/" "binlog_cache_size",32768 "bulk_insert_buffer_size",8388608 "character_set_client","latin1" "character_set_connection","latin1" "character_set_database","latin1" "character_set_filesystem","binary" "character_set_results","latin1" "character_set_server","latin1" "character_set_system","utf8" "character_sets_dir","/usr/share/mysql/charsets/" "collation_connection","latin1_swedish_ci" "collation_database","latin1_swedish_ci" "collation_server","latin1_swedish_ci" "completion_type",0 "concurrent_insert",1 "connect_timeout",10 "datadir","/var/lib/mysql/" "date_format","%Y-%m-%d" "datetime_format","%Y-%m-%d %H:%i:%s" "default_week_format",0 "delay_key_write","ON" "delayed_insert_limit",100 "delayed_insert_timeout",300 "delayed_queue_size",1000 "div_precision_increment",4 "keep_files_on_create","OFF" "engine_condition_pushdown","OFF" "expire_logs_days",0 "flush","OFF" "flush_time",0 "ft_boolean_syntax","+ -><()~*:\"\"&|" "ft_max_word_len",84 "ft_min_word_len",4 "ft_query_expansion_limit",20 "ft_stopword_file","(built-in)" "group_concat_max_len",1024 "have_archive","NO" "have_bdb","YES" "have_blackhole_engine","NO" "have_compress","YES" "have_community_features","NO" "have_profiling","NO" "have_crypt","YES" "have_csv","NO" "have_dynamic_loading","YES" "have_example_engine","NO" "have_federated_engine","NO" "have_geometry","YES" "have_innodb","YES" "have_isam","NO" "have_merge_engine","YES" "have_ndbcluster","DISABLED" "have_openssl","DISABLED" "have_ssl","DISABLED" "have_query_cache","YES" "have_raid","NO" "have_rtree_keys","YES" "have_symlink","DISABLED" "hostname","revere.politico.com" "init_connect","" "init_file","" "init_slave","" "innodb_additional_mem_pool_size",1048576 "innodb_autoextend_increment",8 "innodb_buffer_pool_awe_mem_mb",0 "innodb_buffer_pool_size",134217728 "innodb_checksums","ON" "innodb_commit_concurrency",0 "innodb_concurrency_tickets",500 "innodb_data_file_path","ibdata1:10M:autoextend" "innodb_data_home_dir","" "innodb_adaptive_hash_index","ON" "innodb_doublewrite","ON" "innodb_fast_shutdown",1 "innodb_file_io_threads",4 "innodb_file_per_table","ON" "innodb_flush_log_at_trx_commit",1 "innodb_flush_method","" "innodb_force_recovery",0 "innodb_lock_wait_timeout",50 "innodb_locks_unsafe_for_binlog","OFF" "innodb_log_arch_dir","" "innodb_log_archive","OFF" "innodb_log_buffer_size",1048576 "innodb_log_file_size",5242880 "innodb_log_files_in_group",2 "in
Re: Performance Questions
Andrew, Hi Andrew, Just to endorse what some have said, after reading this thread: 1) concurrency must be ON 2) For your 1 minute task do it in a background thread and use a different OSC. Remember EOF is a single-threaded, single-db-connection stack. If you want high concurrency performance, you cannot just use the default OSC. Use a ERXTaskObjectStoreCoordinatorPool just for tasks, even if it is just a pool of one. Also, if I get a few minutes later or at the weekend, I can eyeball your setup for possible low-hanging fruit if you send the following: A) Send your /etc/my.cnf file to the list, and tell me how much total max memory you want mysql to have - I will take a quick look at it to see if it looks OK. B) send the output of the following SQL statement in a text file: select TABLE_SCHEMA, TABLE_NAME, TABLE_ROWS, (DATA_LENGTH + INDEX_LENGTH)/1024/1024 as SIZE_IN_MB, DATA_LENGTH/1024/1024 as DATA_SIZE_IN_MB, INDEX_LENGTH/1024/1024 as INDEX_SIZE_IN_MB from information_schema.TABLES order by SIZE_IN_MB desc; C) send the output of the following SQL statement in a text file: SHOW VARIABLES; D) Send the output (allschemas.sql) of the following CLI statement: mysqldump --all-databases --opt --no-data > allschemas.sql On Jul 29, 2011, at 10:56 AM, Andrew Kinnie wrote: > That will help, thanks! > > On Jul 29, 2011, at 10:55 AM, Alexis Tual wrote: > >> An example of all that John said is available there thanks to Kieran : >> >> https://github.com/projectwonder/wonder/tree/master/Examples/Misc/BackgroundTasks >> >> Alex >> >> Le 29 juil. 2011 à 16:52, Andrew Kinnie a écrit : >> >>> Thanks. I may give that a try. That was one of the other options I >>> thought of, but was hoping to avoid a significant re-write. >>> >>> >>> >>> On Jul 29, 2011, at 10:44 AM, John & Kim Larson wrote: >>> rather than increasing worker threads, why not just spawn a new Java thread for sending the notifications? That thread can run in the background while you're doing EO stuff and free your app up to do the servicing of requests. If you go down this path, I always pass EOs to other threads as globalIDs to prevent problems. Also, make sure you don't lock the OSC for the app during your work or your app will hang while other threads' ECs wait to get it. If this gets bad enough, use a separate OSC stack and dispose of it when your done. John Sent from my iPhone On Jul 29, 2011, at 9:28 AM, Andrew Kinnie wrote: > Greetings > > I have a deployed app which serves as a push notification server for our > iOS app. It uses a recent ERRest (post WOWODC) to provide access to the > data which is located on a MySQL database (using innoDB). The model has > entities for PushApplication (the iOS app), ApplicationDevice (i.e. an > iOS device which has our iOS app), Notification and has a lookup table > for NotificationType (5 rows). Notification is a message, and there is a > many to many with ApplicationDevice along with a corresponding > device_notification table, as well as ApplicationDeviceNotificationType > to allow particular devices to have particular types of notifications > turned on or off. > > Our app in connected to by our editorial staff via a Cold Fusion app to > send out breaking news alerts as push notifications. I then get (via a > fetch) all the devices which have that particular notification type > (basically 90% of our 20,000+ "installed" applicationDevices), then I > pass that array into a method which makes the connection to Apple and > iterates through the array sending one notification to each device in > turn, then closes the connection. > > It takes approximately 1 minute to send an alert to all 20,000 devices. > > While this happens, some of these devices are getting the push from Apple > (which is crazy fast about it), and some of them are running the app and > the iOS app itself is querying the server for details about the > notification and loading it in. However, if this happens while the push > is still in the process of sending (i.e. within the 1 minute time frame), > the iOS app may be forced to wait for the send process to finish (as many > as 60 seconds presumably. It doesn't happen all that often, because our > app doesn't buzz or makes a sound when it receives a notification, but it > is not ideal. We anticipate using this same app and server for the > Android version, and for the upcoming iPhone update, so the number of > installed devices could increase pretty dramatically. Currently it is > iPad only. > > So, we're trying to figure out what to do about it. Currently the app is > deployed on a CentOS server (single core processor) which also houses the > db, but nothing else. It h
Re: Performance Questions
Le 2011-07-29 à 10:55, Alexis Tual a écrit : > An example of all that John said is available there thanks to Kieran : > > https://github.com/projectwonder/wonder/tree/master/Examples/Misc/BackgroundTasks And hopefully, I will add a REST example in there before the end of the month. > Alex > > Le 29 juil. 2011 à 16:52, Andrew Kinnie a écrit : > >> Thanks. I may give that a try. That was one of the other options I thought >> of, but was hoping to avoid a significant re-write. >> >> >> >> On Jul 29, 2011, at 10:44 AM, John & Kim Larson wrote: >> >>> rather than increasing worker threads, why not just spawn a new Java thread >>> for sending the notifications? That thread can run in the background while >>> you're doing EO stuff and free your app up to do the servicing of requests. >>> >>> If you go down this path, I always pass EOs to other threads as globalIDs >>> to prevent problems. Also, make sure you don't lock the OSC for the app >>> during your work or your app will hang while other threads' ECs wait to get >>> it. If this gets bad enough, use a separate OSC stack and dispose of it >>> when your done. >>> >>> John >>> >>> Sent from my iPhone >>> >>> On Jul 29, 2011, at 9:28 AM, Andrew Kinnie wrote: >>> Greetings I have a deployed app which serves as a push notification server for our iOS app. It uses a recent ERRest (post WOWODC) to provide access to the data which is located on a MySQL database (using innoDB). The model has entities for PushApplication (the iOS app), ApplicationDevice (i.e. an iOS device which has our iOS app), Notification and has a lookup table for NotificationType (5 rows). Notification is a message, and there is a many to many with ApplicationDevice along with a corresponding device_notification table, as well as ApplicationDeviceNotificationType to allow particular devices to have particular types of notifications turned on or off. Our app in connected to by our editorial staff via a Cold Fusion app to send out breaking news alerts as push notifications. I then get (via a fetch) all the devices which have that particular notification type (basically 90% of our 20,000+ "installed" applicationDevices), then I pass that array into a method which makes the connection to Apple and iterates through the array sending one notification to each device in turn, then closes the connection. It takes approximately 1 minute to send an alert to all 20,000 devices. While this happens, some of these devices are getting the push from Apple (which is crazy fast about it), and some of them are running the app and the iOS app itself is querying the server for details about the notification and loading it in. However, if this happens while the push is still in the process of sending (i.e. within the 1 minute time frame), the iOS app may be forced to wait for the send process to finish (as many as 60 seconds presumably. It doesn't happen all that often, because our app doesn't buzz or makes a sound when it receives a notification, but it is not ideal. We anticipate using this same app and server for the Android version, and for the upcoming iPhone update, so the number of installed devices could increase pretty dramatically. Currently it is iPad only. So, we're trying to figure out what to do about it. Currently the app is deployed on a CentOS server (single core processor) which also houses the db, but nothing else. It has 16 GB of RAM. We were considering: 1. Trying to increase the threads the app can create, but I'm not sure that would fix it as much as mask it 2. Trying to run an additional copy of the app to send while the other one handles the incoming client requests, but I am not sure how to accomplish this other than copying the whole project, renaming it, then deploying that. I am also not sure this would fix anything if in fact the issue were locking in the database or jdbc or something of that nature. 3. Seeing if there was something easier, more efficient and less kludgy feeling than either of those. (assuming either of those would work anyway, we have some difficulty testing it without sending out 20,000 push notifications) Anyone have any insight? Andrew ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/the_larsons%40mac.com This email sent to the_lars...@mac.com >> >> ___ >> Do not post admin requests to the list. They will
Re: Performance Questions
That will help, thanks! On Jul 29, 2011, at 10:55 AM, Alexis Tual wrote: > An example of all that John said is available there thanks to Kieran : > > https://github.com/projectwonder/wonder/tree/master/Examples/Misc/BackgroundTasks > > Alex > > Le 29 juil. 2011 à 16:52, Andrew Kinnie a écrit : > >> Thanks. I may give that a try. That was one of the other options I thought >> of, but was hoping to avoid a significant re-write. >> >> >> >> On Jul 29, 2011, at 10:44 AM, John & Kim Larson wrote: >> >>> rather than increasing worker threads, why not just spawn a new Java thread >>> for sending the notifications? That thread can run in the background while >>> you're doing EO stuff and free your app up to do the servicing of requests. >>> >>> If you go down this path, I always pass EOs to other threads as globalIDs >>> to prevent problems. Also, make sure you don't lock the OSC for the app >>> during your work or your app will hang while other threads' ECs wait to get >>> it. If this gets bad enough, use a separate OSC stack and dispose of it >>> when your done. >>> >>> John >>> >>> Sent from my iPhone >>> >>> On Jul 29, 2011, at 9:28 AM, Andrew Kinnie wrote: >>> Greetings I have a deployed app which serves as a push notification server for our iOS app. It uses a recent ERRest (post WOWODC) to provide access to the data which is located on a MySQL database (using innoDB). The model has entities for PushApplication (the iOS app), ApplicationDevice (i.e. an iOS device which has our iOS app), Notification and has a lookup table for NotificationType (5 rows). Notification is a message, and there is a many to many with ApplicationDevice along with a corresponding device_notification table, as well as ApplicationDeviceNotificationType to allow particular devices to have particular types of notifications turned on or off. Our app in connected to by our editorial staff via a Cold Fusion app to send out breaking news alerts as push notifications. I then get (via a fetch) all the devices which have that particular notification type (basically 90% of our 20,000+ "installed" applicationDevices), then I pass that array into a method which makes the connection to Apple and iterates through the array sending one notification to each device in turn, then closes the connection. It takes approximately 1 minute to send an alert to all 20,000 devices. While this happens, some of these devices are getting the push from Apple (which is crazy fast about it), and some of them are running the app and the iOS app itself is querying the server for details about the notification and loading it in. However, if this happens while the push is still in the process of sending (i.e. within the 1 minute time frame), the iOS app may be forced to wait for the send process to finish (as many as 60 seconds presumably. It doesn't happen all that often, because our app doesn't buzz or makes a sound when it receives a notification, but it is not ideal. We anticipate using this same app and server for the Android version, and for the upcoming iPhone update, so the number of installed devices could increase pretty dramatically. Currently it is iPad only. So, we're trying to figure out what to do about it. Currently the app is deployed on a CentOS server (single core processor) which also houses the db, but nothing else. It has 16 GB of RAM. We were considering: 1. Trying to increase the threads the app can create, but I'm not sure that would fix it as much as mask it 2. Trying to run an additional copy of the app to send while the other one handles the incoming client requests, but I am not sure how to accomplish this other than copying the whole project, renaming it, then deploying that. I am also not sure this would fix anything if in fact the issue were locking in the database or jdbc or something of that nature. 3. Seeing if there was something easier, more efficient and less kludgy feeling than either of those. (assuming either of those would work anyway, we have some difficulty testing it without sending out 20,000 push notifications) Anyone have any insight? Andrew ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/the_larsons%40mac.com This email sent to the_lars...@mac.com >> >> ___ >> Do not post admin requests to the list. They will be ignored. >> Webobjects-dev mailing list (W
Re: Performance Questions
well that's a pretty solid point. I had thought it was already "huge" but I
suppose I can exert some pressure now.
On Jul 29, 2011, at 10:54 AM, Pascal Robert wrote:
>
> Le 2011-07-29 à 10:50, Andrew Kinnie a écrit :
>
>> Is that something I would put in the Application constructor? Should that
>> be in a property somewhere?
>
> public Application() {
> ERXApplication.log.info("Welcome to " + name() + " !");
> /* ** put your initialization code in here ** */
> setAllowsConcurrentRequestHandling(true);
> }
>
>> As for the rest, it appears the issue is in the WO app instance. Requests
>> that go to the second instance seem to have normal response times. My IT
>> guy says the entire db is less than 60 MB, so MySQL seems not to be a
>> problem. Apparently we're not using Huge, but there don't seem to be any
>> indications of a memory issue with MySQL.
>
> The database might not be big, but it could be I/O performance problems. And
> your server have tons of RAM, go ahead and tune MySQL right away, why wait
> when you can do it now.
>
>> On Jul 29, 2011, at 10:40 AM, Pascal Robert wrote:
>>
>>> First question: does concurrent request handling
>>> (application.setAllowsConcurrentRequestHandling(true)) is enabled? If not,
>>> just enabled that and you will probably fix most of the performance
>>> issues...
>>>
>>> After that, you need to get metrics to find out where the bottleneck is...
>>> Bandwith? CPU? RAM? The WO app? MySQL? One thing for sure, make sure that
>>> MySQL is at least configured with the "huge" sample configuration file. If
>>> MySQL is not currently using more than 512 MB, you are not using the "huge"
>>> configuration file. Nagios plugins would be useful to gather the metrics.
>>
>
___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Re: Performance Questions
An example of all that John said is available there thanks to Kieran : https://github.com/projectwonder/wonder/tree/master/Examples/Misc/BackgroundTasks Alex Le 29 juil. 2011 à 16:52, Andrew Kinnie a écrit : > Thanks. I may give that a try. That was one of the other options I thought > of, but was hoping to avoid a significant re-write. > > > > On Jul 29, 2011, at 10:44 AM, John & Kim Larson wrote: > >> rather than increasing worker threads, why not just spawn a new Java thread >> for sending the notifications? That thread can run in the background while >> you're doing EO stuff and free your app up to do the servicing of requests. >> >> If you go down this path, I always pass EOs to other threads as globalIDs to >> prevent problems. Also, make sure you don't lock the OSC for the app during >> your work or your app will hang while other threads' ECs wait to get it. If >> this gets bad enough, use a separate OSC stack and dispose of it when your >> done. >> >> John >> >> Sent from my iPhone >> >> On Jul 29, 2011, at 9:28 AM, Andrew Kinnie wrote: >> >>> Greetings >>> >>> I have a deployed app which serves as a push notification server for our >>> iOS app. It uses a recent ERRest (post WOWODC) to provide access to the >>> data which is located on a MySQL database (using innoDB). The model has >>> entities for PushApplication (the iOS app), ApplicationDevice (i.e. an iOS >>> device which has our iOS app), Notification and has a lookup table for >>> NotificationType (5 rows). Notification is a message, and there is a many >>> to many with ApplicationDevice along with a corresponding >>> device_notification table, as well as ApplicationDeviceNotificationType to >>> allow particular devices to have particular types of notifications turned >>> on or off. >>> >>> Our app in connected to by our editorial staff via a Cold Fusion app to >>> send out breaking news alerts as push notifications. I then get (via a >>> fetch) all the devices which have that particular notification type >>> (basically 90% of our 20,000+ "installed" applicationDevices), then I pass >>> that array into a method which makes the connection to Apple and iterates >>> through the array sending one notification to each device in turn, then >>> closes the connection. >>> >>> It takes approximately 1 minute to send an alert to all 20,000 devices. >>> >>> While this happens, some of these devices are getting the push from Apple >>> (which is crazy fast about it), and some of them are running the app and >>> the iOS app itself is querying the server for details about the >>> notification and loading it in. However, if this happens while the push is >>> still in the process of sending (i.e. within the 1 minute time frame), the >>> iOS app may be forced to wait for the send process to finish (as many as 60 >>> seconds presumably. It doesn't happen all that often, because our app >>> doesn't buzz or makes a sound when it receives a notification, but it is >>> not ideal. We anticipate using this same app and server for the Android >>> version, and for the upcoming iPhone update, so the number of installed >>> devices could increase pretty dramatically. Currently it is iPad only. >>> >>> So, we're trying to figure out what to do about it. Currently the app is >>> deployed on a CentOS server (single core processor) which also houses the >>> db, but nothing else. It has 16 GB of RAM. >>> >>> We were considering: >>> >>> 1. Trying to increase the threads the app can create, but I'm not sure that >>> would fix it as much as mask it >>> 2. Trying to run an additional copy of the app to send while the other one >>> handles the incoming client requests, but I am not sure how to accomplish >>> this other than copying the whole project, renaming it, then deploying >>> that. I am also not sure this would fix anything if in fact the issue were >>> locking in the database or jdbc or something of that nature. >>> 3. Seeing if there was something easier, more efficient and less kludgy >>> feeling than either of those. (assuming either of those would work anyway, >>> we have some difficulty testing it without sending out 20,000 push >>> notifications) >>> >>> Anyone have any insight? >>> >>> Andrew >>> ___ >>> Do not post admin requests to the list. They will be ignored. >>> Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) >>> Help/Unsubscribe/Update your Subscription: >>> http://lists.apple.com/mailman/options/webobjects-dev/the_larsons%40mac.com >>> >>> This email sent to the_lars...@mac.com > > ___ > Do not post admin requests to the list. They will be ignored. > Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/webobjects-dev/alexis.tual%40gmail.com > > This email sent to alexis.
Re: Performance Questions
Le 2011-07-29 à 10:50, Andrew Kinnie a écrit :
> Is that something I would put in the Application constructor? Should that be
> in a property somewhere?
public Application() {
ERXApplication.log.info("Welcome to " + name() + " !");
/* ** put your initialization code in here ** */
setAllowsConcurrentRequestHandling(true);
}
> As for the rest, it appears the issue is in the WO app instance. Requests
> that go to the second instance seem to have normal response times. My IT guy
> says the entire db is less than 60 MB, so MySQL seems not to be a problem.
> Apparently we're not using Huge, but there don't seem to be any indications
> of a memory issue with MySQL.
The database might not be big, but it could be I/O performance problems. And
your server have tons of RAM, go ahead and tune MySQL right away, why wait when
you can do it now.
> On Jul 29, 2011, at 10:40 AM, Pascal Robert wrote:
>
>> First question: does concurrent request handling
>> (application.setAllowsConcurrentRequestHandling(true)) is enabled? If not,
>> just enabled that and you will probably fix most of the performance issues...
>>
>> After that, you need to get metrics to find out where the bottleneck is...
>> Bandwith? CPU? RAM? The WO app? MySQL? One thing for sure, make sure that
>> MySQL is at least configured with the "huge" sample configuration file. If
>> MySQL is not currently using more than 512 MB, you are not using the "huge"
>> configuration file. Nagios plugins would be useful to gather the metrics.
>
___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Re: Performance Questions
Thanks. I may give that a try. That was one of the other options I thought of, but was hoping to avoid a significant re-write. On Jul 29, 2011, at 10:44 AM, John & Kim Larson wrote: > rather than increasing worker threads, why not just spawn a new Java thread > for sending the notifications? That thread can run in the background while > you're doing EO stuff and free your app up to do the servicing of requests. > > If you go down this path, I always pass EOs to other threads as globalIDs to > prevent problems. Also, make sure you don't lock the OSC for the app during > your work or your app will hang while other threads' ECs wait to get it. If > this gets bad enough, use a separate OSC stack and dispose of it when your > done. > > John > > Sent from my iPhone > > On Jul 29, 2011, at 9:28 AM, Andrew Kinnie wrote: > >> Greetings >> >> I have a deployed app which serves as a push notification server for our iOS >> app. It uses a recent ERRest (post WOWODC) to provide access to the data >> which is located on a MySQL database (using innoDB). The model has entities >> for PushApplication (the iOS app), ApplicationDevice (i.e. an iOS device >> which has our iOS app), Notification and has a lookup table for >> NotificationType (5 rows). Notification is a message, and there is a many >> to many with ApplicationDevice along with a corresponding >> device_notification table, as well as ApplicationDeviceNotificationType to >> allow particular devices to have particular types of notifications turned on >> or off. >> >> Our app in connected to by our editorial staff via a Cold Fusion app to send >> out breaking news alerts as push notifications. I then get (via a fetch) >> all the devices which have that particular notification type (basically 90% >> of our 20,000+ "installed" applicationDevices), then I pass that array into >> a method which makes the connection to Apple and iterates through the array >> sending one notification to each device in turn, then closes the connection. >> >> It takes approximately 1 minute to send an alert to all 20,000 devices. >> >> While this happens, some of these devices are getting the push from Apple >> (which is crazy fast about it), and some of them are running the app and the >> iOS app itself is querying the server for details about the notification and >> loading it in. However, if this happens while the push is still in the >> process of sending (i.e. within the 1 minute time frame), the iOS app may be >> forced to wait for the send process to finish (as many as 60 seconds >> presumably. It doesn't happen all that often, because our app doesn't buzz >> or makes a sound when it receives a notification, but it is not ideal. We >> anticipate using this same app and server for the Android version, and for >> the upcoming iPhone update, so the number of installed devices could >> increase pretty dramatically. Currently it is iPad only. >> >> So, we're trying to figure out what to do about it. Currently the app is >> deployed on a CentOS server (single core processor) which also houses the >> db, but nothing else. It has 16 GB of RAM. >> >> We were considering: >> >> 1. Trying to increase the threads the app can create, but I'm not sure that >> would fix it as much as mask it >> 2. Trying to run an additional copy of the app to send while the other one >> handles the incoming client requests, but I am not sure how to accomplish >> this other than copying the whole project, renaming it, then deploying that. >> I am also not sure this would fix anything if in fact the issue were >> locking in the database or jdbc or something of that nature. >> 3. Seeing if there was something easier, more efficient and less kludgy >> feeling than either of those. (assuming either of those would work anyway, >> we have some difficulty testing it without sending out 20,000 push >> notifications) >> >> Anyone have any insight? >> >> Andrew >> ___ >> Do not post admin requests to the list. They will be ignored. >> Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) >> Help/Unsubscribe/Update your Subscription: >> http://lists.apple.com/mailman/options/webobjects-dev/the_larsons%40mac.com >> >> This email sent to the_lars...@mac.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Performance Questions
Is that something I would put in the Application constructor? Should that be in a property somewhere? As for the rest, it appears the issue is in the WO app instance. Requests that go to the second instance seem to have normal response times. My IT guy says the entire db is less than 60 MB, so MySQL seems not to be a problem. Apparently we're not using Huge, but there don't seem to be any indications of a memory issue with MySQL. On Jul 29, 2011, at 10:40 AM, Pascal Robert wrote: > First question: does concurrent request handling > (application.setAllowsConcurrentRequestHandling(true)) is enabled? If not, > just enabled that and you will probably fix most of the performance issues... > > After that, you need to get metrics to find out where the bottleneck is... > Bandwith? CPU? RAM? The WO app? MySQL? One thing for sure, make sure that > MySQL is at least configured with the "huge" sample configuration file. If > MySQL is not currently using more than 512 MB, you are not using the "huge" > configuration file. Nagios plugins would be useful to gather the metrics. ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Performance Questions
rather than increasing worker threads, why not just spawn a new Java thread for sending the notifications? That thread can run in the background while you're doing EO stuff and free your app up to do the servicing of requests. If you go down this path, I always pass EOs to other threads as globalIDs to prevent problems. Also, make sure you don't lock the OSC for the app during your work or your app will hang while other threads' ECs wait to get it. If this gets bad enough, use a separate OSC stack and dispose of it when your done. John Sent from my iPhone On Jul 29, 2011, at 9:28 AM, Andrew Kinnie wrote: > Greetings > > I have a deployed app which serves as a push notification server for our iOS > app. It uses a recent ERRest (post WOWODC) to provide access to the data > which is located on a MySQL database (using innoDB). The model has entities > for PushApplication (the iOS app), ApplicationDevice (i.e. an iOS device > which has our iOS app), Notification and has a lookup table for > NotificationType (5 rows). Notification is a message, and there is a many to > many with ApplicationDevice along with a corresponding device_notification > table, as well as ApplicationDeviceNotificationType to allow particular > devices to have particular types of notifications turned on or off. > > Our app in connected to by our editorial staff via a Cold Fusion app to send > out breaking news alerts as push notifications. I then get (via a fetch) all > the devices which have that particular notification type (basically 90% of > our 20,000+ "installed" applicationDevices), then I pass that array into a > method which makes the connection to Apple and iterates through the array > sending one notification to each device in turn, then closes the connection. > > It takes approximately 1 minute to send an alert to all 20,000 devices. > > While this happens, some of these devices are getting the push from Apple > (which is crazy fast about it), and some of them are running the app and the > iOS app itself is querying the server for details about the notification and > loading it in. However, if this happens while the push is still in the > process of sending (i.e. within the 1 minute time frame), the iOS app may be > forced to wait for the send process to finish (as many as 60 seconds > presumably. It doesn't happen all that often, because our app doesn't buzz > or makes a sound when it receives a notification, but it is not ideal. We > anticipate using this same app and server for the Android version, and for > the upcoming iPhone update, so the number of installed devices could increase > pretty dramatically. Currently it is iPad only. > > So, we're trying to figure out what to do about it. Currently the app is > deployed on a CentOS server (single core processor) which also houses the db, > but nothing else. It has 16 GB of RAM. > > We were considering: > > 1. Trying to increase the threads the app can create, but I'm not sure that > would fix it as much as mask it > 2. Trying to run an additional copy of the app to send while the other one > handles the incoming client requests, but I am not sure how to accomplish > this other than copying the whole project, renaming it, then deploying that. > I am also not sure this would fix anything if in fact the issue were locking > in the database or jdbc or something of that nature. > 3. Seeing if there was something easier, more efficient and less kludgy > feeling than either of those. (assuming either of those would work anyway, > we have some difficulty testing it without sending out 20,000 push > notifications) > > Anyone have any insight? > > Andrew > ___ > Do not post admin requests to the list. They will be ignored. > Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/webobjects-dev/the_larsons%40mac.com > > This email sent to the_lars...@mac.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Performance Questions
First question: does concurrent request handling (application.setAllowsConcurrentRequestHandling(true)) is enabled? If not, just enabled that and you will probably fix most of the performance issues... After that, you need to get metrics to find out where the bottleneck is... Bandwith? CPU? RAM? The WO app? MySQL? One thing for sure, make sure that MySQL is at least configured with the "huge" sample configuration file. If MySQL is not currently using more than 512 MB, you are not using the "huge" configuration file. Nagios plugins would be useful to gather the metrics. > Greetings > > I have a deployed app which serves as a push notification server for our iOS > app. It uses a recent ERRest (post WOWODC) to provide access to the data > which is located on a MySQL database (using innoDB). The model has entities > for PushApplication (the iOS app), ApplicationDevice (i.e. an iOS device > which has our iOS app), Notification and has a lookup table for > NotificationType (5 rows). Notification is a message, and there is a many to > many with ApplicationDevice along with a corresponding device_notification > table, as well as ApplicationDeviceNotificationType to allow particular > devices to have particular types of notifications turned on or off. > > Our app in connected to by our editorial staff via a Cold Fusion app to send > out breaking news alerts as push notifications. I then get (via a fetch) all > the devices which have that particular notification type (basically 90% of > our 20,000+ "installed" applicationDevices), then I pass that array into a > method which makes the connection to Apple and iterates through the array > sending one notification to each device in turn, then closes the connection. > > It takes approximately 1 minute to send an alert to all 20,000 devices. > > While this happens, some of these devices are getting the push from Apple > (which is crazy fast about it), and some of them are running the app and the > iOS app itself is querying the server for details about the notification and > loading it in. However, if this happens while the push is still in the > process of sending (i.e. within the 1 minute time frame), the iOS app may be > forced to wait for the send process to finish (as many as 60 seconds > presumably. It doesn't happen all that often, because our app doesn't buzz > or makes a sound when it receives a notification, but it is not ideal. We > anticipate using this same app and server for the Android version, and for > the upcoming iPhone update, so the number of installed devices could increase > pretty dramatically. Currently it is iPad only. > > So, we're trying to figure out what to do about it. Currently the app is > deployed on a CentOS server (single core processor) which also houses the db, > but nothing else. It has 16 GB of RAM. > > We were considering: > > 1. Trying to increase the threads the app can create, but I'm not sure that > would fix it as much as mask it > 2. Trying to run an additional copy of the app to send while the other one > handles the incoming client requests, but I am not sure how to accomplish > this other than copying the whole project, renaming it, then deploying that. > I am also not sure this would fix anything if in fact the issue were locking > in the database or jdbc or something of that nature. > 3. Seeing if there was something easier, more efficient and less kludgy > feeling than either of those. (assuming either of those would work anyway, > we have some difficulty testing it without sending out 20,000 push > notifications) > > Anyone have any insight? > > Andrew > ___ > Do not post admin requests to the list. They will be ignored. > Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/webobjects-dev/probert%40macti.ca > > This email sent to prob...@macti.ca ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Performance Questions
Greetings I have a deployed app which serves as a push notification server for our iOS app. It uses a recent ERRest (post WOWODC) to provide access to the data which is located on a MySQL database (using innoDB). The model has entities for PushApplication (the iOS app), ApplicationDevice (i.e. an iOS device which has our iOS app), Notification and has a lookup table for NotificationType (5 rows). Notification is a message, and there is a many to many with ApplicationDevice along with a corresponding device_notification table, as well as ApplicationDeviceNotificationType to allow particular devices to have particular types of notifications turned on or off. Our app in connected to by our editorial staff via a Cold Fusion app to send out breaking news alerts as push notifications. I then get (via a fetch) all the devices which have that particular notification type (basically 90% of our 20,000+ "installed" applicationDevices), then I pass that array into a method which makes the connection to Apple and iterates through the array sending one notification to each device in turn, then closes the connection. It takes approximately 1 minute to send an alert to all 20,000 devices. While this happens, some of these devices are getting the push from Apple (which is crazy fast about it), and some of them are running the app and the iOS app itself is querying the server for details about the notification and loading it in. However, if this happens while the push is still in the process of sending (i.e. within the 1 minute time frame), the iOS app may be forced to wait for the send process to finish (as many as 60 seconds presumably. It doesn't happen all that often, because our app doesn't buzz or makes a sound when it receives a notification, but it is not ideal. We anticipate using this same app and server for the Android version, and for the upcoming iPhone update, so the number of installed devices could increase pretty dramatically. Currently it is iPad only. So, we're trying to figure out what to do about it. Currently the app is deployed on a CentOS server (single core processor) which also houses the db, but nothing else. It has 16 GB of RAM. We were considering: 1. Trying to increase the threads the app can create, but I'm not sure that would fix it as much as mask it 2. Trying to run an additional copy of the app to send while the other one handles the incoming client requests, but I am not sure how to accomplish this other than copying the whole project, renaming it, then deploying that. I am also not sure this would fix anything if in fact the issue were locking in the database or jdbc or something of that nature. 3. Seeing if there was something easier, more efficient and less kludgy feeling than either of those. (assuming either of those would work anyway, we have some difficulty testing it without sending out 20,000 push notifications) Anyone have any insight? Andrew ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
