http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/configure_client_server_event_messaging.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/configure_client_server_event_messaging.html.md.erb b/geode-docs/developing/events/configure_client_server_event_messaging.html.md.erb deleted file mode 100644 index ebd4a3a..0000000 --- a/geode-docs/developing/events/configure_client_server_event_messaging.html.md.erb +++ /dev/null @@ -1,81 +0,0 @@ ---- -title: Configuring Client/Server Event Messaging ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -You can receive events from your servers for server-side cache events and query result changes. - -<a id="receiving_events_from_servers__section_F21FB253CCC244708CB953B6D5866A91"></a> -For cache updates, you can configure to receive entry keys and values or just entry keys, with the data retrieved lazily when requested. The queries are run continuously against server cache events, with the server sending the deltas for your query result sets. - -Before you begin, set up your client/server installation and configure and program your basic event messaging. - -Servers receive updates for all entry events in their client's client regions. - -To receive entry events in the client from the server: - -1. Set the client pool `subscription-enabled` to true. See [<pool>](../../reference/topics/client-cache.html#cc-pool). -2. Program the client to register interest in the entries you need. - - **Note:** - This must be done through the API. - - Register interest in all keys, a key list, individual keys, or by comparing key strings to regular expressions. By default, no entries are registered to receive updates. Specify whether the server is to send values with entry update events. Interest registration is only available through the API. - - 1. Get an instance of the region where you want to register interest. - 2. Use the regions's `registerInterest`\* methods to specify the entries you want. Examples: - - ``` pre - // Register interest in a single key and download its entry - // at this time, if it is available in the server cache - Region region1 = . . . ; - region1.registerInterest("key-1"); - - // Register Interest in a List of Keys but do not do an initial bulk load - // do not send values for creater/update events - just send key with invalidation - Region region2 = . . . ; - List list = new ArrayList(); - list.add("key-1"); - list.add("key-2"); - list.add("key-3"); - list.add("key-4"); - region2.registerInterest(list, InterestResultPolicy.NONE, false); - - // Register interest in all keys and download all available keys now - Region region3 = . . . ; - region3.registerInterest("ALL_KEYS", InterestResultPolicy.KEYS); - - // Register Interest in all keys matching a regular expression - Region region1 = . . . ; - region1.registerInterestRegex("[a-zA-Z]+_[0-9]+"); - ``` - - You can call the register interest methods multiple times for a single region. Each interest registration adds to the serverâs list of registered interest criteria for the client. So if a client registers interest in key âAâ, then registers interest in regular expression "B\*", the server will send updates for all entries with key âAâ or key beginning with the letter âBâ. - - 3. For highly available event messaging, configure server redundancy. See [Configuring Highly Available Servers](configuring_highly_available_servers.html). - 4. To have events enqueued for your clients during client downtime, configure durable client/server messaging. - 5. Write any continuous queries (CQs) that you want to run to receive continuously streaming updates to client queries. CQ events do not update the client cache. If you have dependencies between CQs and/or interest registrations, so that you want the two types of subscription events to arrive as closely together on the client, use a single server pool for everything. Using different pools can lead to time differences in the delivery of events because the pools might use different servers to process and deliver the event messages. - -- **[Configuring Highly Available Servers](../../developing/events/configuring_highly_available_servers.html)** - -- **[Implementing Durable Client/Server Messaging](../../developing/events/implementing_durable_client_server_messaging.html)** - -- **[Tuning Client/Server Event Messaging](../../developing/events/tune_client_server_event_messaging.html)** - -
http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/configure_multisite_event_messaging.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/configure_multisite_event_messaging.html.md.erb b/geode-docs/developing/events/configure_multisite_event_messaging.html.md.erb deleted file mode 100644 index 5756652..0000000 --- a/geode-docs/developing/events/configure_multisite_event_messaging.html.md.erb +++ /dev/null @@ -1,39 +0,0 @@ ---- -title: Configuring Multi-Site (WAN) Event Queues ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -In a multi-site (WAN) installation, Geode uses gateway sender queues to distribute events for regions that are configured with a gateway sender. AsyncEventListeners also use an asynchronous event queue to distribute events for configured regions. This section describes additional options for configuring the event queues that are used by gateway senders or AsyncEventListener implementations. - -<a id="configure_multisite_event_messaging__section_1BBF77E166E84F7CA110385FD03D8453"></a> -Before you begin, set up your multi-site (WAN) installation or configure asynchronous event queues and AsyncEventListener implementations. See [Configuring a Multi-site (WAN) System](../../topologies_and_comm/multi_site_configuration/setting_up_a_multisite_system.html#setting_up_a_multisite_system) or [Implementing an AsyncEventListener for Write-Behind Cache Event Handling](implementing_write_behind_event_handler.html#implementing_write_behind_cache_event_handling). - -- **[Persisting an Event Queue](../../developing/events/configuring_highly_available_gateway_queues.html)** - - You can configure a gateway sender queue or an asynchronous event queue to persist data to disk similar to the way in which replicated regions are persisted. - -- **[Configuring Dispatcher Threads and Order Policy for Event Distribution](../../developing/events/configuring_gateway_concurrency_levels.html)** - - By default, Geode uses multiple dispatcher threads to process region events simultaneously in a gateway sender queue for distribution between sites, or in an asynchronous event queue for distributing events for write-behind caching. With serial queues, you can also configure the ordering policy for dispatching those events. - -- **[Conflating Events in a Queue](../../developing/events/conflate_multisite_gateway_queue.html)** - - Conflating a queue improves distribution performance. When conflation is enabled, only the latest queued value is sent for a particular key. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/configure_p2p_event_messaging.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/configure_p2p_event_messaging.html.md.erb b/geode-docs/developing/events/configure_p2p_event_messaging.html.md.erb deleted file mode 100644 index a733ce7..0000000 --- a/geode-docs/developing/events/configure_p2p_event_messaging.html.md.erb +++ /dev/null @@ -1,50 +0,0 @@ ---- -title: Configuring Peer-to-Peer Event Messaging ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -You can receive events from distributed system peers for any region that is not a local region. Local regions receive only local cache events. - -<a id="configuring_event_distribution__section_7D5B1F0C0EF24E58BB3C335CB4EA9A3C"></a> -Peer distribution is done according to the region's configuration. - -- Replicated regions always receive all events from peers and require no further configuration. Replicated regions are configured using the `REPLICATE` region shortcut settings. -- For non-replicated regions, decide whether you want to receive all entry events from the distributed cache or only events for the data you have stored locally. To configure: - - To receive all events, set the `subscription-attributes` `interest-policy` to `all`: - - ``` pre - <region-attributes> - <subscription-attributes interest-policy="all"/> - </region-attributes> - ``` - - - To receive events just for the data you have stored locally, set the `subscription-attributes` `interest-policy` to `cache-content` or do not set it (`cache-content` is the default): - - ``` pre - <region-attributes> - <subscription-attributes interest-policy="cache-content"/> - </region-attributes> - ``` - - For partitioned regions, this only affects the receipt of events, as the data is stored according to the region partitioning. Partitioned regions with interest policy of `all` can create network bottlenecks, so if you can, run listeners in every member that hosts the partitioned region data and use the `cache-content` interest policy. - -**Note:** -You can also configure Regions using the gfsh command-line interface. See [Region Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_EF03119A40EE492984F3B6248596E1DD). - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/configuring_gateway_concurrency_levels.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/configuring_gateway_concurrency_levels.html.md.erb b/geode-docs/developing/events/configuring_gateway_concurrency_levels.html.md.erb deleted file mode 100644 index e59d3b4..0000000 --- a/geode-docs/developing/events/configuring_gateway_concurrency_levels.html.md.erb +++ /dev/null @@ -1,158 +0,0 @@ ---- -title: Configuring Dispatcher Threads and Order Policy for Event Distribution ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -By default, Geode uses multiple dispatcher threads to process region events simultaneously in a gateway sender queue for distribution between sites, or in an asynchronous event queue for distributing events for write-behind caching. With serial queues, you can also configure the ordering policy for dispatching those events. - -By default, a gateway sender queue or asynchronous event queue uses 5 dispatcher threads per queue. This provides support for applications that have the ability to process queued events concurrently for distribution to another Geode site or listener. If your application does not require concurrent distribution, or if you do not have enough resources to support the requirements of multiple dispatcher threads, then you can configure a single dispatcher thread to process a queue. - -- [Using Multiple Dispatcher Threads to Process a Queue](configuring_gateway_concurrency_levels.html#concept_6C52A037E39E4FD6AE4C6A982A4A1A85__section_20E8EFCE89EB4DC7AA822D03C8E0F470) -- [Performance and Memory Considerations](configuring_gateway_concurrency_levels.html#concept_6C52A037E39E4FD6AE4C6A982A4A1A85__section_C4C83B5C0FDD4913BA128365EE7E4E35) -- [Configuring the Ordering Policy for Serial Queues](configuring_gateway_concurrency_levels.html#concept_6C52A037E39E4FD6AE4C6A982A4A1A85__section_4835BA30CDFD4B658BD2576F6BC2E23F) -- [ExamplesâConfiguring Dispatcher Threads and Ordering Policy for a Serial Gateway Sender Queue](configuring_gateway_concurrency_levels.html#concept_6C52A037E39E4FD6AE4C6A982A4A1A85__section_752F08F9064B4F67A80DA0A994671EA0) - -## <a id="concept_6C52A037E39E4FD6AE4C6A982A4A1A85__section_20E8EFCE89EB4DC7AA822D03C8E0F470" class="no-quick-link"></a>Using Multiple Dispatcher Threads to Process a Queue - -When multiple dispatcher threads are configured for a parallel queue, Geode simply uses multiple threads to process the contents of each individual queue. The total number of queues that are created is still determined by the number of Geode members that host the region. - -When multiple dispatcher threads are configured for a serial queue, Geode creates an additional copy of the queue for each thread on each member that hosts the queue. To obtain the maximum throughput, increase the number of dispatcher threads until your network is saturated. - -The following diagram illustrates a serial gateway sender queue that is configured with multiple dispatcher threads. -<img src="../../images/MultisiteConcurrency_WAN_Gateway.png" id="concept_6C52A037E39E4FD6AE4C6A982A4A1A85__image_093DAC58EBEE456485562C92CA79899F" class="image" width="624" /> - -## <a id="concept_6C52A037E39E4FD6AE4C6A982A4A1A85__section_C4C83B5C0FDD4913BA128365EE7E4E35" class="no-quick-link"></a>Performance and Memory Considerations - -When a serial gateway sender or an asynchronous event queue uses multiple dispatcher threads, consider the following: - -- Queue attributes are repeated for each copy of the queue that is created for a dispatcher thread. That is, each concurrent queue points to the same disk store, so the same disk directories are used. If persistence is enabled and overflow occurs, the threads that insert entries into the queues compete for the disk. This applies to application threads and dispatcher threads, so it can affect application performance. -- The `maximum-queue-memory` setting applies to each copy of the serial queue. If you configure 10 dispatcher threads and the maximum queue memory is set to 100MB, then the total maximum queue memory for the queue is 1000MB on each member that hosts the queue. - -## <a id="concept_6C52A037E39E4FD6AE4C6A982A4A1A85__section_4835BA30CDFD4B658BD2576F6BC2E23F" class="no-quick-link"></a>Configuring the Ordering Policy for Serial Queues - -When using multiple `dispatcher-threads` (greater than 1) with a serial event queue, you can also configure the `order-policy` that those threads use to distribute events from the queue. The valid order policy values are: - -- **key (default)**. All updates to the same key are distributed in order. Geode preserves key ordering by placing all updates to the same key in the same dispatcher thread queue. You typically use key ordering when updates to entries have no relationship to each other, such as for an application that uses a single feeder to distribute stock updates to several other systems. -- **thread**. All region updates from a given thread are distributed in order. Geode preserves thread ordering by placing all region updates from the same thread into the same dispatcher thread queue. In general, use thread ordering when updates to one region entry affect updates to another region entry. -- **partition**. All region events that share the same partitioning key are distributed in order. Specify partition ordering when applications use a [PartitionResolver](/releases/latest/javadoc/org/apache/geode/cache/PartitionResolver.html) to implement [custom partitioning](../partitioned_regions/using_custom_partition_resolvers.html). With partition ordering, all entries that share the same "partitioning key" (RoutingObject) are placed into the same dispatcher thread queue. - -You cannot configure the `order-policy` for a parallel event queue, because parallel queues cannot preserve event ordering for regions. Only the ordering of events for a given partition (or in a given queue of a distributed region) can be preserved. - -## <a id="concept_6C52A037E39E4FD6AE4C6A982A4A1A85__section_752F08F9064B4F67A80DA0A994671EA0" class="no-quick-link"></a>ExamplesâConfiguring Dispatcher Threads and Ordering Policy for a Serial Gateway Sender Queue - -To increase the number of dispatcher threads and set the ordering policy for a serial gateway sender, use one of the following mechanisms. - -- **cache.xml configuration** - - ``` pre - <cache> - <gateway-sender id="NY" parallel="false" - remote-distributed-system-id="1" - enable-persistence="true" - disk-store-name="gateway-disk-store" - maximum-queue-memory="200" - dispatcher-threads=7 order-policy="key"/> - ... - </cache> - ``` - -- **Java API configuration** - - ``` pre - Cache cache = new CacheFactory().create(); - - GatewaySenderFactory gateway = cache.createGatewaySenderFactory(); - gateway.setParallel(false); - gateway.setPersistenceEnabled(true); - gateway.setDiskStoreName("gateway-disk-store"); - gateway.setMaximumQueueMemory(200); - gateway.setDispatcherThreads(7); - gateway.setOrderPolicy(OrderPolicy.KEY); - GatewaySender sender = gateway.create("NY", "1"); - sender.start(); - ``` - -- **gfsh:** - - ``` pre - gfsh>create gateway-sender -d="NY" - --parallel=false - --remote-distributed-system-id="1" - --enable-persistence=true - --disk-store-name="gateway-disk-store" - --maximum-queue-memory=200 - --dispatcher-threads=7 - --order-policy="key" - ``` - -The following examples show how to set dispatcher threads and ordering policy for an asynchronous event queue: - -- **cache.xml configuration** - - ``` pre - <cache> - <async-event-queue id="sampleQueue" persistent="true" - disk-store-name="async-disk-store" parallel="false" - dispatcher-threads=7 order-policy="key"> - <async-event-listener> - <class-name>MyAsyncEventListener</class-name> - <parameter name="url"> - <string>jdbc:db2:SAMPLE</string> - </parameter> - <parameter name="username"> - <string>gfeadmin</string> - </parameter> - <parameter name="password"> - <string>admin1</string> - </parameter> - </async-event-listener> - </async-event-queue> - ... - </cache> - ``` - -- **Java API configuration** - - ``` pre - Cache cache = new CacheFactory().create(); - AsyncEventQueueFactory factory = cache.createAsyncEventQueueFactory(); - factory.setPersistent(true); - factory.setDiskStoreName("async-disk-store"); - factory.setParallel(false); - factory.setDispatcherThreads(7); - factory.setOrderPolicy(OrderPolicy.KEY); - AsyncEventListener listener = new MyAsyncEventListener(); - AsyncEventQueue sampleQueue = factory.create("customerWB", listener); - ``` - - Entry updates in the current, in-process batch are not eligible for conflation. - -- **gfsh:** - - ``` pre - gfsh>create async-event-queue --id="sampleQueue" --persistent=true - --disk-store="async-disk-store" --parallel=false - --dispatcher-threads=7 order-policy="key" - --listener=myAsycEventListener - --listener-param=url#jdbc:db2:SAMPLE - --listener-param=username#gfeadmin - --listener-param=password#admin1 - ``` - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/configuring_highly_available_gateway_queues.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/configuring_highly_available_gateway_queues.html.md.erb b/geode-docs/developing/events/configuring_highly_available_gateway_queues.html.md.erb deleted file mode 100644 index 28339e2..0000000 --- a/geode-docs/developing/events/configuring_highly_available_gateway_queues.html.md.erb +++ /dev/null @@ -1,119 +0,0 @@ ---- -title: Persisting an Event Queue ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -You can configure a gateway sender queue or an asynchronous event queue to persist data to disk similar to the way in which replicated regions are persisted. - -<a id="configuring_highly_available_gateway_queues__section_7EB2A7E38B074AAAA06D22C59687CB8A"></a> -Persisting a queue provides high availability for the event messaging that the sender performs. For example, if a persistent gateway sender queue exits for any reason, when the member that hosts the sender restarts it automatically reloads the queue and resumes sending messages. If an asynchronous event queue exits for any reason, write-back caching can resume where it left off when the queue is brought back online. -Geode persists an event queue if you set the `enable-persistence` attribute to true. The queue is persisted to the disk store specified in the queue's `disk-store-name` attribute, or to the default disk store if you do not specify a store name. - -You must configure the event queue to use persistence if you are using persistent regions. The use of non-persistent event queues with persistent regions is not supported. - -When you enable persistence for a queue, the `maximum-queue-memory` attribute determines how much memory the queue can consume before it overflows to disk. By default, this value is set to 100MB. - -**Note:** -If you configure a parallel queue and/or you configure multiple dispatcher threads for a queue, the values that are defined in the `maximum-queue-memory` and `disk-store-name` attributes apply to each instance of the queue. - -In the example below the gateway sender queue uses "diskStoreA" for persistence and overflow, and the queue has a maximum queue memory of 100MB: - -- XML example: - - ``` pre - <cache> - <gateway-sender id="persistedsender1" parallel="false" - remote-distributed-system-id="1" - enable-persistence="true" - disk-store-name="diskStoreA" - maximum-queue-memory="100"/> - ... - </cache> - ``` - -- API example: - - ``` pre - Cache cache = new CacheFactory().create(); - - GatewaySenderFactory gateway = cache.createGatewaySenderFactory(); - gateway.setParallel(false); - gateway.setPersistenceEnabled(true); - gateway.setDiskStoreName("diskStoreA"); - gateway.setMaximumQueueMemory(100); - GatewaySender sender = gateway.create("persistedsender1", "1"); - sender.start(); - ``` - -- gfsh: - - ``` pre - gfsh>create gateway-sender --id="persistedsender1 --parallel=false - --remote-distributed-system-id=1 --enable-persistence=true --disk-store-name=diskStoreA - --maximum-queue-memory=100 - ``` - -If you were to configure 10 dispatcher threads for the serial gateway sender, then the total maximum memory for the gateway sender queue would be 1000MB on each Geode member that hosted the sender, because Geode creates a separate copy of the queue per thread.. - -The following example shows a similar configuration for an asynchronous event queue: - -- XML example: - - ``` pre - <cache> - <async-event-queue id="persistentAsyncQueue" persistent="true" - disk-store-name="diskStoreA" parallel="true"> - <async-event-listener> - <class-name>MyAsyncEventListener</class-name> - <parameter name="url"> - <string>jdbc:db2:SAMPLE</string> - </parameter> - <parameter name="username"> - <string>gfeadmin</string> - </parameter> - <parameter name="password"> - <string>admin1</string> - </parameter> - </async-event-listener> - </async-event-queue> - ... - </cache> - ``` - -- API example: - - ``` pre - Cache cache = new CacheFactory().create(); - AsyncEventQueueFactory factory = cache.createAsyncEventQueueFactory(); - factory.setPersistent(true); - factory.setDiskStoreName("diskStoreA"); - factory.setParallel(true); - AsyncEventListener listener = new MyAsyncEventListener(); - AsyncEventQueue persistentAsyncQueue = factory.create("customerWB", listener); - ``` - -- gfsh: - - ``` pre - gfsh>create async-event-queue --id="persistentAsyncQueue" --persistent=true - --disk-store="diskStoreA" --parallel=true --listener=MyAsyncEventListener - --listener-param=url#jdbc:db2:SAMPLE --listener-param=username#gfeadmin --listener-param=password#admin1 - ``` - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/configuring_highly_available_servers.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/configuring_highly_available_servers.html.md.erb b/geode-docs/developing/events/configuring_highly_available_servers.html.md.erb deleted file mode 100644 index 48cd174..0000000 --- a/geode-docs/developing/events/configuring_highly_available_servers.html.md.erb +++ /dev/null @@ -1,55 +0,0 @@ ---- -title: Configuring Highly Available Servers ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -<a id="configuring_highly_available_servers__section_7EB2A7E38B074AAAA06D22C59687CB8A"></a> -With highly-available servers, one of the backups steps in and takes over messaging with no interruption in service if the client's primary server crashes. - -To configure high availability, set the `subscription-redundancy` in the client's pool configuration. This setting indicates the number of secondary servers to use. For example: - -``` pre -<!-- Run one secondary server --> -<pool name="red1" subscription-enabled="true" subscription-redundancy="1"> - <locator host="nick" port="41111"/> - <locator host="nora" port="41111"/> -</pool> -``` - -``` pre -<!-- Use all available servers as secondaries. One is primary, the rest are secondaries --> -<pool name="redX" subscription-enabled="true" subscription-redundancy="-1"> - <locator host="nick" port="41111"/> - <locator host="nora" port="41111"/> -</pool> -``` - -When redundancy is enabled, secondary servers maintain queue backups while the primary server pushes events to the client. If the primary server fails, one of the secondary servers steps in as primary to provide uninterrupted event messaging to the client. - -The following table describes the different values for the subscription-redundancy setting: - -| subscription-redundancy | Description | -|-------------------------|--------------------------------------------------------------------------------| -| 0 | No secondary servers are configured, so high availability is disabled. | -| > 0 | Sets the precise number of secondary servers to use for backup to the primary. | -| -1 | Every server that is not the primary is to be used as a secondary. | - -- **[Highly Available Client/Server Event Messaging](../../developing/events/ha_event_messaging_whats_next.html)** - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/conflate_multisite_gateway_queue.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/conflate_multisite_gateway_queue.html.md.erb b/geode-docs/developing/events/conflate_multisite_gateway_queue.html.md.erb deleted file mode 100644 index 3088ac3..0000000 --- a/geode-docs/developing/events/conflate_multisite_gateway_queue.html.md.erb +++ /dev/null @@ -1,130 +0,0 @@ ---- -title: Conflating Events in a Queue ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -Conflating a queue improves distribution performance. When conflation is enabled, only the latest queued value is sent for a particular key. - -<a id="conflate_multisite_gateway_queue__section_294AD2E2328E4D6B8D6A73966F7B3B14"></a> -**Note:** -Do not use conflation if your receiving applications depend on the specific ordering of entry modifications, or if they need to be notified of every change to an entry. - -Conflation is most useful when a single entry is updated frequently, but other sites only need to know the current value of the entry (rather than the value of each update). When an update is added to a queue that has conflation enabled, if there is already an update message in the queue for the entry key, then the existing message assumes the value of the new update and the new update is dropped, as shown here for key A. - -<img src="../../images/MultiSite-4.gif" id="conflate_multisite_gateway_queue__image_27219DAAB6D643348641389DBAEA1E94" class="image" /> - -**Note:** -This method of conflation is different from the one used for server-to-client subscription queue conflation and peer-to-peer distribution within a distributed system. - -## <a id="conflate_multisite_gateway_queue__section_207FA6BF0F734F9A91EAACB136F8D6B5" class="no-quick-link"></a>ExamplesâConfiguring Conflation for a Gateway Sender Queue - -To enable conflation for a gateway sender queue, use one of the following mechanisms: - -- **cache.xml configuration** - - ``` pre - <cache> - <gateway-sender id="NY" parallel="true" - remote-distributed-system-id="1" - enable-persistence="true" - disk-store-name="gateway-disk-store" - enable-batch-conflation="true"/> - ... - </cache> - ``` - -- **Java API configuration** - - ``` pre - Cache cache = new CacheFactory().create(); - - GatewaySenderFactory gateway = cache.createGatewaySenderFactory(); - gateway.setParallel(true); - gateway.setPersistenceEnabled(true); - gateway.setDiskStoreName("gateway-disk-store"); - gateway.setBatchConflationEnabled(true); - GatewaySender sender = gateway.create("NY", "1"); - sender.start(); - ``` - - Entry updates in the current, in-process batch are not eligible for conflation. - -- **gfsh:** - - ``` pre - gfsh>create gateway-sender --id="NY" --parallel=true - --remote-distributed-system-id="1" - --enable-persistence=true - --disk-store-name="gateway-disk-store" - --enable-batch-conflation=true - ``` - -The following examples show how to configure conflation for an asynchronous event queue: - -- **cache.xml configuration** - - ``` pre - <cache> - <async-event-queue id="sampleQueue" persistent="true" - disk-store-name="async-disk-store" parallel="false" - enable-batch-conflation="true"> - <async-event-listener> - <class-name>MyAsyncEventListener</class-name> - <parameter name="url"> - <string>jdbc:db2:SAMPLE</string> - </parameter> - <parameter name="username"> - <string>gfeadmin</string> - </parameter> - <parameter name="password"> - <string>admin1</string> - </parameter> - </async-event-listener> - </async-event-queue> - ... - </cache> - ``` - -- **Java API configuration** - - ``` pre - Cache cache = new CacheFactory().create(); - AsyncEventQueueFactory factory = cache.createAsyncEventQueueFactory(); - factory.setPersistent(true); - factory.setDiskStoreName("async-disk-store"); - factory.setParallel(false); - factory.setBatchConflationEnabled(true); - AsyncEventListener listener = new MyAsyncEventListener(); - AsyncEventQueue sampleQueue = factory.create("customerWB", listener); - ``` - - Entry updates in the current, in-process batch are not eligible for conflation. - -- **gfsh:** - - ``` pre - gfsh>create async-event-queue --id="sampleQueue" --persistent=true - --disk-store="async-disk-store" --parallel="false" - --listener=myAsyncEventListener - --listener-param=url#jdbc:db2:SAMPLE - --listener-param=username#gfeadmin - --listener-param=password#admin1 - ``` - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/conflate_server_subscription_queue.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/conflate_server_subscription_queue.html.md.erb b/geode-docs/developing/events/conflate_server_subscription_queue.html.md.erb deleted file mode 100644 index 8a16a64..0000000 --- a/geode-docs/developing/events/conflate_server_subscription_queue.html.md.erb +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: Conflate the Server Subscription Queue ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -<a id="conflate_the_server_subscription_queue__section_1791DFB89502480EB57F81D16AC0EBAC"></a> -Conflating the server subscription queue can save space in the server and time in message processing. - -Enable conflation at the server level in the server region configuration: - -``` pre -<region ... > - <region-attributes enable-subscription-conflation="true" /> -</region> -``` - -Override the server setting as needed, on a per-client basis, in the clientâs `gemfire.properties`: - -``` pre -conflate-events=false -``` - -Valid `conflate-events` settings are: -- `server`, which uses the server settings -- `true`, which conflates everything sent to the client -- `false`, which does not conflate anything sent to this client - -Conflation can both improve performance and reduce the amount of memory required on the server for queuing. The client receives only the latest available update in the queue for a particular entry key. Conflation is disabled by default. - -Conflation is particularly useful when a single entry is updated often and the intermediate updates donât require processing by the client. With conflation, if an entry is updated and there is already an update in the queue for its key, the existing update is removed and the new update is placed at the end of the queue. Conflation is only done on messages that are not in the process of being sent to the client. - -<img src="../../images/ClientServerAdvancedTopics-7.gif" id="conflate_the_server_subscription_queue__image_FA77FD2857464D17BF2ED5B3CC62687A" class="image" /> - -**Note:** -This method of conflation is different from the one used for multi-site gateway sender queue conflation. It is the same as the method used for the conflation of peer-to-peer distribution messages within a single distributed system. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/event_handler_overview.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/event_handler_overview.html.md.erb b/geode-docs/developing/events/event_handler_overview.html.md.erb deleted file mode 100644 index 22a053f..0000000 --- a/geode-docs/developing/events/event_handler_overview.html.md.erb +++ /dev/null @@ -1,40 +0,0 @@ ---- -title: Implementing Geode Event Handlers ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -You can specify event handlers for region and region entry operations and for administrative events. - -- **[Implementing Cache Event Handlers](implementing_cache_event_handlers.html)** - - Depending on your installation and configuration, cache events can come from local operations, peers, servers, and remote sites. Event handlers register their interest in one or more events and are notified when the events occur. - -- **[Implementing an AsyncEventListener for Write-Behind Cache Event Handling](implementing_write_behind_event_handler.html)** - - An `AsyncEventListener` asynchronously processes batches of events after they have been applied to a region. You can use an `AsyncEventListener` implementation as a write-behind cache event handler to synchronize region updates with a database. - -- **[How to Safely Modify the Cache from an Event Handler Callback](writing_callbacks_that_modify_the_cache.html)** - - Event handlers are synchronous. If you need to change the cache or perform any other distributed operation from event handler callbacks, be careful to avoid activities that might block and affect your overall system performance. - -- **[Cache Event Handler Examples](cache_event_handler_examples.html)** - - Some examples of cache event handlers. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/filtering_multisite_events.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/filtering_multisite_events.html.md.erb b/geode-docs/developing/events/filtering_multisite_events.html.md.erb deleted file mode 100644 index 505cd9c..0000000 --- a/geode-docs/developing/events/filtering_multisite_events.html.md.erb +++ /dev/null @@ -1,126 +0,0 @@ ---- -title: Filtering Events for Multi-Site (WAN) Distribution ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -You can optionally create gateway sender and/or gateway receiver filters to control which events are queued and distributed to a remote site, or to modify the data stream that is transmitted between Geode sites. - -You can implement and deploy two different types of filter for multi-site events: - -- `GatewayEventFilter`. A `GatewayEventFilter` implementation determines whether a region event is placed in a gateway sender queue and/or whether an event in a gateway queue is distributed to a remote site. You can optionally add one or more `GatewayEventFilter` implementations to a gateway sender, etiher in the `cache.xml` configuration file or using the Java API. - - Geode makes a synchronous call to the filter's `beforeEnqueue` method before it places a region event in the gateway sender queue. The filter returns a boolean value that specifies whether the event should be added to the queue. - - Geode asynchronously calls the filter's `beforeTransmit` method to determine whether the gateway sender dispatcher thread should distribute the event to a remote gateway receiver. - - For events that are distributed to another site, Geode calls the listener's `afterAcknowledgement` method to indicate that is has received an ack from the remote site after the event was received. - -- GatewayTransportFilter. Use a `GatewayTransportFilter` implementation to process the TCP stream that sends a batch of events that is distributed from one Geode cluster to another over a WAN. A `GatewayTransportFilter` is typically used to perform encryption or compression on the data that distributed. You install the same `GatewayTransportFilter` implementation on both a gateway sender and gateway receiver. - - When a gateway sender processes a batch of events for distribution, Geode delivers the stream to the `getInputStream` method of a configured `GatewayTransportFilter` implementation. The filter processes and returns the stream, which is then transmitted to the gateway receiver. When the gateway receiver receives the batch, Geode calls the `getOutputStream` method of a configured filter, which again processes and returns the stream so that the events can be applied in the local cluster. - -## <a id="topic_E97BB68748F14987916CD1A50E4B4542__section_E20B4A8A98FD4EDAAA8C14B8059AA7F7" class="no-quick-link"></a>Configuring Multi-Site Event Filters - -You install a `GatewayEventFilter` implementation to a configured gateway sender in order to decide which events are queued and distributed. You install a `GatewayTransportFilter` implementation to both a gateway sender and a gateway receiver to process the stream of batched events that are distributed between two sites: - -- **XML example** - - ``` pre - <cache> - <gateway-sender id="remoteA" parallel="true" remote-distributed-system-id="1"> - <gateway-event-filter> - <class-name>org.apache.geode.util.SampleEventFilter</class-name> - <parameter name="param1"> - <string>"value1"</string> - </parameter> - </gateway-event-filter> - <gateway-transport-filter> - <class-name>org.apache.geode.util.SampleTransportFilter</class-name> - <parameter name="param1"> - <string>"value1"</string> - </parameter> - </gateway-transport-filter> - </gateway-sender> - </cache> - ``` - - ``` pre - <cache> - ... - <gateway-receiver start-port="1530" end-port="1551"> - <gateway-transport-filter> - <class-name>org.apache.geode.util.SampleTransportFilter</class-name> - <parameter name="param1"> - <string>"value1"</string> - </parameter> - </gateway-transport-filter> - </gateway-receiver> - </cache> - ``` - -- **gfsh example** - - ``` pre - gfsh>create gateway-sender --id=remoteA --parallel=true --remote-distributed-id="1" - --gateway-event-filter=org.apache.geode.util.SampleEventFilter - --gateway-transport-filter=org.apache.geode.util.SampleTransportFilter - ``` - - See [create gateway-sender](../../tools_modules/gfsh/command-pages/create.html#topic_hg2_bjz_ck). - - ``` pre - gfsh>create gateway-receiver --start-port=1530 --end-port=1551 \ - --gateway-transport-filter=org.apache.geode.util.SampleTransportFilter - ``` - - **Note:** - You cannot specify parameters and values for the Java class you specify with the `--gateway-transport-filter` option. - - See [create gateway-receiver](../../tools_modules/gfsh/command-pages/create.html#topic_a4x_pb1_dk). - -- **API example** - - ``` pre - Cache cache = new CacheFactory().create(); - - GatewayEventFilter efilter = new SampleEventFilter(); - GatewayTransportFilter tfilter = new SampleTransportFilter(); - - GatewaySenderFactory gateway = cache.createGatewaySenderFactory(); - gateway.setParallel(true); - gateway.addGatewayEventFilter(efilter); - gateway.addTransportFilter(tfilter); - GatewaySender sender = gateway.create("remoteA", "1"); - sender.start(); - ``` - - ``` pre - Cache cache = new CacheFactory().create(); - - GatewayTransportFilter tfilter = new SampleTransportFilter(); - - GatewayReceiverFactory gateway = cache.createGatewayReceiverFactory(); - gateway.setStartPort(1530); - gateway.setEndPort(1551); - gateway.addTransportFilter(tfilter); - GatewayReceiver receiver = gateway.create(); - receiver.start(); - ``` - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/ha_event_messaging_whats_next.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/ha_event_messaging_whats_next.html.md.erb b/geode-docs/developing/events/ha_event_messaging_whats_next.html.md.erb deleted file mode 100644 index 40077ea..0000000 --- a/geode-docs/developing/events/ha_event_messaging_whats_next.html.md.erb +++ /dev/null @@ -1,95 +0,0 @@ ---- -title: Highly Available Client/Server Event Messaging ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -<a id="ha_event_messaging_whats_next__section_F163917311E8478399D1DD273E8BCDF5"></a> -With server redundancy, each pool has a primary server and some number of secondaries. The primaries and secondaries are assigned on a per-pool basis and are generally spread out for load balancing, so a single client with multiple pools may have primary queues in more than one server. - -The primary server pushes events to clients and the secondaries maintain queue backups. If the primary server fails, one of the secondaries becomes primary to provide uninterrupted event messaging. - -For example, if there are six servers running and `subscription-redundancy` is set to two, one server is the primary, two servers are secondary, and the remaining three do not actively participate in HA for the client. If the primary server fails, the system assigns one of the secondaries as the new primary and attempts to add another server to the secondary pool to retain the initial redundancy level. If no new secondary server is found, then the redundancy level is not satisfied but the failover procedure completes successfully. As soon as another secondary is available, it is added. - -When high availability is enabled: - -- The primary server sends event messages to the clients. -- Periodically, the clients send received messages to the server and the server removes the sent messages from its queues. -- Periodically, the primary server synchronizes with its secondaries, notifying them of messages that can be discarded because they have already been sent and received. There is a lag in notification, so the secondary servers remain only roughly synchronized with the primary. Secondary queues contain all messages that are contained in the primary queue plus possibly a few messages that have already been sent to clients. -- In the case of primary server failure, one of the secondaries becomes the primary and begins sending event messages from its queues to the clients. Immediately after failover, the new primary usually resends some messages that were already sent by the old primary. The client recognizes these as duplicates and discards them. - -In stage 1 of this figure, the primary sends an event message to the client and a synchronization message to its secondary. By stage 2, the secondary and client have updated their queue and message tracking information. If the primary failed at stage two, the secondary would start sending event messages from its queue beginning with message A10. The client would discard the resend of message A10 and then process subsequent messages as usual. -<img src="../../images/ClientServerAdvancedTopics-5.gif" alt="High Availability Messaging: Server to Client and Primary Server to Secondary Server" id="ha_event_messaging_whats_next__image_8947A42EDEF74911BAB55B79ED8DA984" class="image" /> - -## <a id="ha_event_messaging_whats_next__section_741052B413F24F47A14F5B7D7955F0AA" class="no-quick-link"></a>Change Server Queue Synchronization Frequency - -By default, the primary server sends queue synchronization messages to the secondaries every second. You can change this interval with the `gfsh alter runtime` command - -Set the interval for queue synchronization messages as follows: - -- gfsh: - - ``` pre - gfsh>alter runtime --message-sync-interval=2 - ``` - -- XML: - - ``` pre - <!-- Set sync interval to 2 seconds --> - <cache ... message-sync-interval="2" /> - ``` - -- Java: - - ``` pre - cache = CacheFactory.create(); - cache.setMessageSyncInterval(2); - ``` - -The ideal setting for this interval depends in large part on your application behavior. These are the benefits of shorter and longer interval settings: - -- A shorter interval requires less memory in the secondary servers because it reduces queue buildup between synchronizations. In addition, fewer old messages in the secondary queues means reduced message re-sends after a failover. These considerations are most important for systems with high data update rates. -- A longer interval requires fewer distribution messages between the primary and secondary, which benefits overall system performance. - -## <a id="ha_event_messaging_whats_next__section_DF51950D30154A58818F6AD777BB3090" class="no-quick-link"></a>Set Frequency of Orphan Removal from the Secondary Queues - -Usually, all event messages are removed from secondary subscription queues based on the primary's synchronization messages. Occasionally, however, some messages are orphaned in the secondary queues. For example, if a primary fails in the middle of sending a synchronization message to its secondaries, some secondaries might receive the message and some might not. If the failover goes to a secondary that did receive the message, the system will have secondary queues holding messages that are no longer in the primary queue. The new primary will never synchronize on these messages, leaving them orphaned in the secondary queues. - -To make sure these messages are eventually removed, the secondaries expire all messages that have been enqueued longer than the time indicated by the servers' `message-time-to-live`. - -Set the time-to-live as follows: - -- XML: - - ``` pre - <!-- Set message ttl to 5 minutes --> - <cache-server port="41414" message-time-to-live="300" /> - ``` - -- Java: - - ``` pre - Cache cache = ...; - CacheServer cacheServer = cache.addCacheServer(); - cacheServer.setPort(41414); - cacheServer.setMessageTimeToLive(200); - cacheServer.start(); - ``` - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/how_cache_events_work.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/how_cache_events_work.html.md.erb b/geode-docs/developing/events/how_cache_events_work.html.md.erb deleted file mode 100644 index e54371a..0000000 --- a/geode-docs/developing/events/how_cache_events_work.html.md.erb +++ /dev/null @@ -1,71 +0,0 @@ ---- -title: Peer-to-Peer Event Distribution ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -When a region or entry operation is performed, Geode distributes the associated events in the distributed system according to system and cache configurations. - -<a id="how_cache_events_work__section_7864A275FDB549FD8E2D046DD59CB9F4"></a> -Install a cache listener for a region in each system member that needs to receive notification of region and entry changes. - -## <a id="how_cache_events_work__section_CACE500A00214CD88CE232D22899263B" class="no-quick-link"></a>Events in a Partitioned Region - -A distributed operation follows this sequence in a partitioned region: - -1. Apply the operation to the cache with the primary data entry, if appropriate. -2. Do the distribution based on the subscription-attributes interest-policy of the other members. -3. Invoke any listeners in the caches that receive the distribution. -4. Invoke the listener in the cache with the primary data entry. - -In the following figure: - -1. An API call in member M1 creates an entry. -2. The partitioned region creates the new entry in the cache in M2. M2, the holder of the primary copy, drives the rest of the procedure. -3. These two operations occur simultaneously: - - The partitioned region creates a secondary copy of the entry in the cache in M3. Creating the secondary copy does not invoke the listener on M3. - - M2 distributes the event to M4. This distribution to the other members is based on their interest policies. M4 has an interest-policy of all , so it receives notification of all events anywhere in the region. Since M1 and M3 have an interest-policy of cache-content , and this event does not affect any pre-existing entry in their local caches, they do not receive the event. - -4. The cache listener on M4 handles the notification of the remote event on M2. -5. Once everything on the other members has completed successfully, the original create operation on M2 succeeds and invokes the cache listener on M2. - -<img src="../../images/Events-2.gif" id="how_cache_events_work__image_E5E187C14A774144B85FA7B636239DBE" class="image" /> - -## <a id="how_cache_events_work__section_FACF58272C824907BA020B1727427D7A" class="no-quick-link"></a>Events in a Distributed Region - -A distributed operation follows this sequence in a distributed region: - -1. Apply the operation to the local cache, if appropriate. -2. Invoke the local listeners. -3. Do the distribution. -4. Each member that receives the distribution carries out its own operation in response, which invokes any local listeners. - -In the following figure: - -1. An entry is created through a direct API call on member M1. -2. The create invokes the cache listener on M1. -3. M1 distributes the event to the other members. -4. M2 and M3 apply the remote change through their own local operations. -5. M3 does a create, but M2 does an update, because the entry already existed in its cache. -6. The cache listener on M2 receives callbacks for the local update. Since there is no cache listener on M3, the callbacks from the create on M3 are not handled. An API call in member M1 creates an entry. - -<img src="../../images/Events-3.gif" id="how_cache_events_work__image_A24D6182B2A840D1843EBD4686966EEF" class="image" /> - -## <a id="how_cache_events_work__section_B4DCA51DDF7F44699E7355277172BEF0" class="no-quick-link"></a>Managing Events in Multi-threaded Applications - -For partitioned regions, Geode guarantees ordering of events across threads, but for distributed regions it doesnât. For multi-threaded applications that create distributed regions, you need to use your application synchronization to make sure that one operation completes before the next one begins. Distribution through the distributed-no-ack queue can work with multiple threads if you set the `conserve-sockets` attribute to true. Then the threads share one queue, preserving the order of the events in distributed regions. Different threads can invoke the same listener, so if you allow different threads to send events, it can result in concurrent invocations of the listener. This is an issue only if the threads have some shared state - if they are incrementing a serial number, for example, or adding their events to a log queue. Then you need to make your code thread safe. http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/how_client_server_distribution_works.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/how_client_server_distribution_works.html.md.erb b/geode-docs/developing/events/how_client_server_distribution_works.html.md.erb deleted file mode 100644 index be7e4f2..0000000 --- a/geode-docs/developing/events/how_client_server_distribution_works.html.md.erb +++ /dev/null @@ -1,137 +0,0 @@ ---- -title: Client-to-Server Event Distribution ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -Clients and servers distribute events according to client activities and according to interest registered by the client in server-side cache changes. - -<a id="how_client_server_distribution_works__section_F1070B06B3344D1CA7309934FCE097B9"></a> -When the client updates its cache, changes to client regions are automatically forwarded to the server side. The server-side update is then propagated to the other clients that are connected and have subscriptions enabled. The server does not return the update to the sending client. - -The update is passed to the server and then passed, with the value, to every other client that has registered interest in the entry key. This figure shows how a clientâs entry updates are propagated. - -<img src="../../images_svg/client_server_event_dist.svg" id="how_client_server_distribution_works__image_66AB57EEDC154962B32F7951667F4656" class="image" /> - -The figure shows the following process: - -1. Entry X is updated or created in Region A through a direct API call on Client1. -2. The update to the region is passed to the pool named in the region. -3. The pool propagates the event to the cache server, where the region is updated. -4. The server member distributes the event to its peers and also places it into the subscription queue for Client2 because that client has previously registered interest in entry X. -5. The event for entry X is sent out of the queue to Client2. When this happens is indeterminate. - -Client to server distribution uses the client pool connections to send updates to the server. Any region with a named pool automatically forwards updates to the server. Client cache modifications pass first through a client `CacheWriter`, if one is defined, then to the server through the client pool, and then finally to the client cache itself. A cache writer, either on the client or server side, may abort the operation. - -| Change in Client Cache | Effect on Server Cache | -|-----------------------------------------------------|-----------------------------------------------------------------------------------------------------------| -| Entry create or update | Creation or update of entry. | -| Distributed entry destroy | Entry destroy. The destroy call is propagated to the server even if the entry is not in the client cache. | -| Distributed region destroy/clear (distributed only) | Region destroy/clear | - -**Note:** -Invalidations on the client side are not forwarded to the server. - -## <a id="how_client_server_distribution_works__section_A16562611E094C88B12BC149D5EEEEBA" class="no-quick-link"></a>Server-to-Client Event Distribution - -The server automatically sends entry modification events only for keys in which the client has registered interest. In the interest registration, the client indicates whether to send new values or just invalidations for the server-side entry creates and updates. If invalidation is used, the client then updates the values lazily as needed. - -This figure shows the complete event subscription event distribution for interest registrations, with value receipt requested (receiveValues=true) and without. - -<img src="../../images_svg/server_client_event_dist.svg" id="how_client_server_distribution_works__image_7FD1450B9D58429F860400801EDFDCAE" class="image" /> - -<table> -<colgroup> -<col width="50%" /> -<col width="50%" /> -</colgroup> -<thead> -<tr class="header"> -<th>Change in Server Cache</th> -<th>Effect on Client Cache</th> -</tr> -</thead> -<tbody> -<tr class="odd"> -<td>Entry create/update</td> -<td>For subscriptions with <code class="ph codeph">receiveValues</code> set to true, entry create or update. -<p></p> -For subscriptions with <code class="ph codeph">receiveValues</code> set to false, entry invalidate if the entry already exists in the client cache; otherwise, no effect. The next client get for the entry is forwarded to the server.</td> -</tr> -<tr class="even"> -<td>Entry invalidate/destroy (distributed only)</td> -<td>Entry invalidate/destroy</td> -</tr> -<tr class="odd"> -<td>Region destroy/clear (distributed only)</td> -<td>Region destroy or local region clear</td> -</tr> -</tbody> -</table> - -Server-side distributed operations are all operations that originate as a distributed operation in the server or one of its peers. Region invalidation in the server is not forwarded to the client. - -**Note:** -To maintain a unified set of data in your servers, do not do local entry invalidation in your server regions. - -## <a id="how_client_server_distribution_works__section_34613292788D4FA4AB730045FB9A405A" class="no-quick-link"></a>Server-to-Client Message Tracking - -The server uses an asynchronous messaging queue to send events to its clients. Every event in the queue originates in an operation performed by a thread in a client, a server, or an application in the serverâs or some other distributed system. The event message has a unique identifier composed of the originating threadâs ID combined with its memberâs distributed system member ID, and the sequential ID of the operation. So the event messages originating in any single thread can be grouped and ordered by time from lowest sequence ID to highest. Servers and clients track the highest sequential ID for each member thread ID. - -A single client thread receives and processes messages from the server, tracking received messages to make sure it does not process duplicate sends. It does this using the process IDs from originating threads. - -<img src="../../images_svg/client_server_message_tracking.svg" id="how_client_server_distribution_works__image_F4F9D13252E14F11AD63240AED39191A" class="image" /> - -The clientâs message tracking list holds the highest sequence ID of any message received for each originating thread. The list can become quite large in systems where there are many different threads coming and going and doing work on the cache. After a thread dies, its tracking entry is not needed. To avoid maintaining tracking information for threads that have died, the client expires entries that have had no activity for more than the `subscription-message-tracking-timeout`. - -## <a id="how_client_server_distribution_works__section_99E436C569F3422AA842AA74F73A6B36" class="no-quick-link"></a>Client Interest Registration on the Server - -The system processes client interest registration following these steps: - -1. The entries in the client region that may be affected by this registration are silently destroyed. Other keys are left alone. - - For the `registerInterest` method, the system destroys all of the specified keys, leaving other keys in the client region alone. So if you have a client region with keys A, B, and C and you register interest in the key list A, B, at the start of the `registerInterest` operation, the system destroys keys A and B in the client cache but does not touch key C. - - For the `registerInterestRegex` method, the system silently destroys all keys in the client region. - -2. The interest specification is sent to the server, where it is added to the clientâs interest list. The list can specify entries that are not in the server region at the time interest is registered. -3. If a bulk load is requested in the call's `InterestResultPolicy` parameter, before control is returned to the calling method, the server sends all data that currently satisfies the interest specification. The client's region is updated automatically with the downloaded data. If the server region is partitioned, the entire partitioned region is used in the bulk load. Otherwise, only the serverâs local cache region is used. The interest results policy options are: - - KEYSâThe client receives a bulk load of all available keys matching the interest registration criteria. - - KEYS\_VALUESâThe client receives a bulk load of all available keys and values matching the interest registration criteria. This is the default interest result policy. - - NONEâThe client does not receive any immediate bulk loading. - -Once interest is registered, the server continually monitors region activities and sends events to its clients that match the interest. - -- No events are generated by the register interest calls, even if they load values into the client cache. -- The server maintains the union of all of the interest registrations, so if a client registers interest in key âAâ, then registers interest in regular expression "B\*", the server will send updates for all entries with key âAâ or key beginning with the letter âBâ. -- The server maintains the interest registration list separate from the region. The list can contain specifications for entries that are not currently in the server region. -- The `registerInterestRegex` method uses the standard `java.util.regex` methods to parse the key specification. - -## <a id="how_client_server_distribution_works__section_928BB60066414BEB9FAA7FB3120334A3" class="no-quick-link"></a>Server Failover - -When a server hosting a subscription queue fails, the queueing responsibilities pass to another server. How this happens depends on whether the new server is a secondary server. In any case, all failover activities are carried out automatically by the Geode system. - -- **Non-HA failover:** The client fails over without high availability if it is not configured for redundancy or if all secondaries also fail before new secondaries can be initialized. As soon as it can attach to a server, the client goes through an automatic reinitialization process. In this process, the failover code on the client side silently destroys all entries of interest to the client and refetches them from the new server, essentially reinitializing the client cache from the new serverâs cache. For the notify all configuration, this clears and reloads all of the entries for the client regions that are connected to the server. For notify by subscription, it clears and reloads only the entries in the region interest lists. To reduce failover noise, the events caused by the local entry destruction and refetching are blocked by the failover code and do not reach the client cache listeners. Because of this, your clients could receive some out-of-sequence events during and af ter a server failover. For example, entries that exist on the failed server and not on its replacement are destroyed and never recreated during a failover. Because the destruction events are blocked, the client ends up with entries removed from its cache with no associated destroy events. -- **HA failover:** If your client pool is configured with redundancy and a secondary server is available at the time the primary fails, the failover is invisible to the client. The secondary server resumes queueing activities as soon as the primary loss is detected. The secondary might resend a few events, which are discarded automatically by the client message tracking activities. - - **Note:** - There is a very small potential for message loss during HA server failover. The risk is not present for failover to secondaries that have fully initialized their subscription queue data. The risk is extremely low in healthy systems that use at least two secondary servers. The risk is higher in unstable systems where servers often fail and where secondaries do not have time to initialize their subscription queue data before becoming primaries. To minimize the risk, the failover logic chooses the longest-lived secondary as the new primary. - - - **Note:** - Redundancy management is handled by the client, so when a durable client is disconnected from the server, client event redundancy is not maintained. Even if the servers fail one at a time, so that running clients have time to fail over and pick new secondary servers, an offline durable client cannot fail over. As a result, the client loses its queued messages. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/how_events_work.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/how_events_work.html.md.erb b/geode-docs/developing/events/how_events_work.html.md.erb deleted file mode 100644 index 2ac899a..0000000 --- a/geode-docs/developing/events/how_events_work.html.md.erb +++ /dev/null @@ -1,111 +0,0 @@ ---- -title: How Events Work ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -Members in your Geode distributed system receive cache updates from other members through cache events. The other members can be peers to the member, clients or servers or other distributed systems. - -## <a id="how_events_work__section_6C75098DDBB84944ADE57F2088330D5A" class="no-quick-link"></a>Events Features - -These are the primary features of Geode events: - -- Content-based events -- Asynchronous event notifications with conflation -- Synchronous event notifications for low latency -- High availability through redundant messaging queues -- Event ordering and once and only-once delivery -- Distributed event notifications -- Durable subscriptions -- Continuous querying - -## <a id="how_events_work__section_F5F0E506652940DFBA0D6B7AAA72E3EF" class="no-quick-link"></a>Types of Events - -There are two categories of events and event handlers. - -- Cache events in the caching API are used by applications with a cache. Cache events provide detail-level notification for changes to your data. Continuous query events are in this category. -- Administrative events in the administration API are used by administrative applications without caches. - -Both kinds of events can be generated by a single member operation. - -**Note:** -You can handle one of these categories of events in a single system member. You cannot handle both cache and administrative events in a single member. - -Because Geode maintains the order of administrative events and the order of cache events separately, using cache events and administrative events in a single process can cause unexpected results. - -## <a id="how_events_work__section_4BCDB22AB927478EBF1035B0DE230DD3" class="no-quick-link"></a>Event Cycle - -The following steps describe the event cycle: - -1. An operation begins, such as data put or a cache close. -2. The operation execution generates these objects: - - An object of type `Operation` that describes the method that triggered the event. - - An event object that describes the event, such as the member and region where the operation originated. - -3. The event handlers that can handle the event are called and passed the event objects. Different event types require different handler types in different locations. If there is no matching event handler, that does not change the effect of the operation, which happens as usual. -4. When the handler receives the event, it triggers the handlerâs callback method for this event. The callback method can hand off the event object as input to another method. Depending on the type of event handler, the callbacks can be triggered before or after the operation. The timing depends on the event handler, not on the event itself. - **Note:** - For transactions, after-operation listeners receive the events after the transaction has committed. - -5. If the operation is distributed, so that it causes follow-on operations in other members, those operations generate their own events, which can be handled by their listeners in the same way. - -## <a id="how_events_work__section_E48ECC8A1B39411AA23D17BA0C05517E" class="no-quick-link"></a>Event Objects - -Event objects come in several types, depending on the operation. Some operations generate multiple objects of different types. All event objects contain data describing the event, and each event type carries slightly different kinds of data appropriate to its matching operation. An event object is stable. For example, its content does not change if you pass it off to a method on another thread. - -For cache events, the event object describes the operation performed in the local cache. If the event originated remotely, it describes the local application of the remote entry operation, not the remote operation itself. The only exception is when the local region has an empty data policy; then the event carries the information for the remote (originating) cache operation. - -## <a id="how_events_work__section_2EA59E9F7203433A8AD248C499D61BF4" class="no-quick-link"></a>Event Distribution - -After a member processes an event in its local cache, it distributes it to remote caches according to the member's configuration and the configurations of the remote caches. For example, if a client updates its cache, the update is forwarded to the client's server. The server distributes the update to its peers and forwards it to any other clients according to their interest in the data entry. If the server system is part of a multi-site deployment and the data region is configured to use a gateway sender, then the gateway sender also forwards the update to a remote site, where the update is further distributed and propagated. - -## <a id="how_events_work__section_C18D3CA923FB427AA01DD811589D63C0" class="no-quick-link"></a>Event Handlers and Region Data Storage - -You can configure a region for no local data storage and still send and receive events for the region. Conversely, if you store data in the region, the cache is updated with data from the event regardless of whether you have any event handlers installed. - -## <a id="how_events_work__section_22EB4B9E6C4445F898DB64A769780460" class="no-quick-link"></a>Multiple Listeners - -When multiple listeners are installed, as can be done with cache listeners, the listeners are invoked sequentially in the order they were added to the region or cache. Listeners are executed one at a time. So, unless you program a listener to pass off processing to another thread, you can use one listener's work in later listeners. - -## <a id="how_events_work__section_C4758D7E2CA2498A87315DE903A07AE4" class="no-quick-link"></a>Event Ordering - -During a cache operation, event handlers are called at various stages of the operation. Some event handlers are called before a region update and some are called after the region update operation. Depending on the type of event handler being called, the event handler can receive the events in-order or out-of-order in which they are applied on Region. - -- `CacheWriter` and `AsyncEventListener` always receive events in the order in which they are applied on region. -- `CacheListener` and `CqListener` can receive events in a different order than the order in which they were applied on the region. - -**Note:** -An `EntryEvent` contains both the old value and the new value of the entry, which helps to indicate the value that was replaced by the cache operation on a particular key. - -- **[Peer-to-Peer Event Distribution](../../developing/events/how_cache_events_work.html)** - - When a region or entry operation is performed, Geode distributes the associated events in the distributed system according to system and cache configurations. - -- **[Client-to-Server Event Distribution](../../developing/events/how_client_server_distribution_works.html)** - - Clients and servers distribute events according to client activities and according to interest registered by the client in server-side cache changes. - -- **[Multi-Site (WAN) Event Distribution](../../developing/events/how_multisite_distribution_works.html)** - - Geode distributes a subset of cache events between distributed systems, with a minimum impact on each system's performance. Events are distributed only for regions that you configure to use a gateway sender for distribution. - -- **[List of Event Handlers and Events](../../developing/events/list_of_event_handlers_and_events.html)** - - Geode provides many types of events and event handlers to help you manage your different data and application needs. - -
