SAMZA-1629: Integration Tests for all samples of hello-samza Author: Sanil Jain <snj...@linkedin.com>
Reviewers: Jagadish<jagad...@apache.org> Closes #44 from Sanil15/latest Project: http://git-wip-us.apache.org/repos/asf/samza-hello-samza/repo Commit: http://git-wip-us.apache.org/repos/asf/samza-hello-samza/commit/f25c37d3 Tree: http://git-wip-us.apache.org/repos/asf/samza-hello-samza/tree/f25c37d3 Diff: http://git-wip-us.apache.org/repos/asf/samza-hello-samza/diff/f25c37d3 Branch: refs/heads/latest Commit: f25c37d3ce451fcb89501932309c14d7db853e2d Parents: 108b6d5 Author: Sanil Jain <snj...@linkedin.com> Authored: Mon Nov 12 15:12:15 2018 -0800 Committer: Jagadish <jvenkatra...@linkedin.com> Committed: Mon Nov 12 15:12:15 2018 -0800 ---------------------------------------------------------------------- README.md | 14 + bin/deploy.sh | 2 +- build.gradle | 11 +- gradle.properties | 2 +- pom.xml | 24 +- .../cookbook/RemoteTableJoinExample.java | 2 +- .../cookbook/StreamTableJoinExample.java | 14 + .../samza/examples/cookbook/data/AdClick.java | 11 + .../application/WikipediaApplication.java | 38 +- .../test/TestSamzaCookBookExamples.java | 179 ++++ .../samza/examples/test/utils/TestUtils.java | 93 ++ .../test/TestWikipediaApplication.java | 82 ++ .../wikipedia/task/test/TestWikipediaTask.java | 70 ++ src/test/resources/WikinewsEditEvents.txt | 104 +++ src/test/resources/WikipediaEditEvents.txt | 882 +++++++++++++++++++ src/test/resources/WikitionaryEditEvents.txt | 9 + 16 files changed, 1527 insertions(+), 10 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/samza-hello-samza/blob/f25c37d3/README.md ---------------------------------------------------------------------- diff --git a/README.md b/README.md index 7bd700d..81c7624 100644 --- a/README.md +++ b/README.md @@ -84,6 +84,20 @@ deploy/samza/bin/run-app.sh --config-factory=org.apache.samza.config.factories.P Once the jobs are started, you can use the same _kafka-console-consumer.sh_ command as in the high-level API Wikipedia example to check out the output of the statistics. +#### 4. Run all the examples as Integration Test + +Every example above are ran with a few messages as Integration test using TestRunner API. You can find all the testing samples in [src/test/java](https://github.com/apache/samza-hello-samza/tree/master/src/test/java). To run it use: + +``` +mvn clean package +``` + +Run Single example as test use: + +``` +mvn test -Dtest=<ClassName> +``` + ### Contribution To start contributing on [Hello Samza](http://samza.apache.org/startup/hello-samza/latest/) first read [Rules](http://samza.apache.org/contribute/rules.html) and [Contributor Corner](https://cwiki.apache.org/confluence/display/SAMZA/Contributor%27s+Corner). Notice that [Hello Samza](http://samza.apache.org/startup/hello-samza/latest/) git repository does not support git pull request. http://git-wip-us.apache.org/repos/asf/samza-hello-samza/blob/f25c37d3/bin/deploy.sh ---------------------------------------------------------------------- diff --git a/bin/deploy.sh b/bin/deploy.sh index 08c0601..3c3ada2 100755 --- a/bin/deploy.sh +++ b/bin/deploy.sh @@ -23,4 +23,4 @@ base_dir=`pwd` mvn clean package mkdir -p $base_dir/deploy/samza -tar -xvf $base_dir/target/hello-samza-1.0.1-SNAPSHOT-dist.tar.gz -C $base_dir/deploy/samza +tar -xvf $base_dir/target/hello-samza-1.0.0-SNAPSHOT-dist.tar.gz -C $base_dir/deploy/samza http://git-wip-us.apache.org/repos/asf/samza-hello-samza/blob/f25c37d3/build.gradle ---------------------------------------------------------------------- diff --git a/build.gradle b/build.gradle index a897372..80dafea 100644 --- a/build.gradle +++ b/build.gradle @@ -36,6 +36,14 @@ repositories { maven { url "https://repository.apache.org/content/groups/public" } } + +idea { + module { + sourceDirs += file('src/main/java') + testSourceDirs += file('src/test/java') + } +} + // a configuration for dependencies that need exploding into package configurations { explode @@ -48,11 +56,12 @@ dependencies { compile(group: 'org.schwering', name: 'irclib', version: '1.10') compile(group: 'org.apache.samza', name: 'samza-api', version: "$SAMZA_VERSION") compile(group: 'org.apache.samza', name: 'samza-kv_2.11', version: "$SAMZA_VERSION") + compile(group: 'org.apache.samza', name: 'samza-test_2.11', version: "$SAMZA_VERSION") compile(group: 'org.apache.samza', name: 'samza-kafka_2.11', version: "$SAMZA_VERSION") compile(group: 'org.apache.samza', name: 'samza-kv-rocksdb_2.11', version: "$SAMZA_VERSION") compile(group: 'org.apache.samza', name: 'samza-azure', version: "$SAMZA_VERSION") + testCompile(group: 'junit', name: 'junit', version: "4.12") explode (group: 'org.apache.samza', name: 'samza-shell', ext: 'tgz', classifier: 'dist', version: "$SAMZA_VERSION") - runtime(group: 'org.apache.samza', name: 'samza-core_2.11', version: "$SAMZA_VERSION") runtime(group: 'org.apache.samza', name: 'samza-log4j', version: "$SAMZA_VERSION") runtime(group: 'org.apache.samza', name: 'samza-shell', version: "$SAMZA_VERSION") http://git-wip-us.apache.org/repos/asf/samza-hello-samza/blob/f25c37d3/gradle.properties ---------------------------------------------------------------------- diff --git a/gradle.properties b/gradle.properties index bfc8582..5da3d5d 100644 --- a/gradle.properties +++ b/gradle.properties @@ -17,7 +17,7 @@ * under the License. */ -SAMZA_VERSION=1.0.1 +SAMZA_VERSION=1.0.0 KAFKA_VERSION=0.11.0.2 HADOOP_VERSION=2.6.1 http://git-wip-us.apache.org/repos/asf/samza-hello-samza/blob/f25c37d3/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index 167ea1e..e82647c 100644 --- a/pom.xml +++ b/pom.xml @@ -27,7 +27,7 @@ under the License. <groupId>org.apache.samza</groupId> <artifactId>hello-samza</artifactId> - <version>1.0.1-SNAPSHOT</version> + <version>1.0.0-SNAPSHOT</version> <packaging>jar</packaging> <name>Samza Example</name> <description> @@ -37,6 +37,11 @@ under the License. <dependencies> <dependency> + <groupId>junit</groupId> + <artifactId>junit</artifactId> + <version>4.12</version> + </dependency> + <dependency> <groupId>org.apache.samza</groupId> <artifactId>samza-api</artifactId> <version>${samza.version}</version> @@ -53,6 +58,11 @@ under the License. </dependency> <dependency> <groupId>org.apache.samza</groupId> + <artifactId>samza-test_2.11</artifactId> + <version>${samza.version}</version> + </dependency> + <dependency> + <groupId>org.apache.samza</groupId> <artifactId>samza-log4j</artifactId> <version>${samza.version}</version> </dependency> @@ -158,7 +168,7 @@ under the License. <properties> <!-- maven specific properties --> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> - <samza.version>1.0.1-SNAPSHOT</samza.version> + <samza.version>1.0.0</samza.version> <hadoop.version>2.6.1</hadoop.version> </properties> @@ -228,6 +238,7 @@ under the License. <configuration> <excludes> <exclude>*.patch</exclude> + <exclude>*.txt</exclude> <exclude>**/target/**</exclude> <exclude>*.json</exclude> <exclude>.vagrant/**</exclude> @@ -244,6 +255,7 @@ under the License. <exclude>**/gradle/**</exclude> <exclude>**/gradlew*</exclude> <exclude>**/build/**</exclude> + <exclude>**/src/test/resources/**</exclude> </excludes> </configuration> </plugin> @@ -292,6 +304,14 @@ under the License. </executions> </plugin> </plugins> + <resources> + <resource> + <directory>src/test/resources</directory> + <includes> + <include>**/*txt</include> + </includes> + </resource> + </resources> </build> <profiles> http://git-wip-us.apache.org/repos/asf/samza-hello-samza/blob/f25c37d3/src/main/java/samza/examples/cookbook/RemoteTableJoinExample.java ---------------------------------------------------------------------- diff --git a/src/main/java/samza/examples/cookbook/RemoteTableJoinExample.java b/src/main/java/samza/examples/cookbook/RemoteTableJoinExample.java index 2418ae6..4eb4a56 100644 --- a/src/main/java/samza/examples/cookbook/RemoteTableJoinExample.java +++ b/src/main/java/samza/examples/cookbook/RemoteTableJoinExample.java @@ -184,7 +184,7 @@ public class RemoteTableJoinExample implements StreamApplication { } } - static class StockPrice implements Serializable { + public static class StockPrice implements Serializable { public final String symbol; public final Double close; http://git-wip-us.apache.org/repos/asf/samza-hello-samza/blob/f25c37d3/src/main/java/samza/examples/cookbook/StreamTableJoinExample.java ---------------------------------------------------------------------- diff --git a/src/main/java/samza/examples/cookbook/StreamTableJoinExample.java b/src/main/java/samza/examples/cookbook/StreamTableJoinExample.java index 4fb6254..96196fa 100644 --- a/src/main/java/samza/examples/cookbook/StreamTableJoinExample.java +++ b/src/main/java/samza/examples/cookbook/StreamTableJoinExample.java @@ -18,6 +18,7 @@ */ package samza.examples.cookbook; +import java.util.Objects; import org.apache.samza.application.StreamApplication; import org.apache.samza.application.descriptors.StreamApplicationDescriptor; import org.apache.samza.operators.KV; @@ -155,6 +156,19 @@ public class StreamTableJoinExample implements StreamApplication { this.company = company; this.pageId = pageId; } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (o == null || getClass() != o.getClass()) { + return false; + } + EnrichedPageView that = (EnrichedPageView) o; + return Objects.equals(userId, that.userId) && Objects.equals(company, that.company) && Objects.equals(pageId, + that.pageId); + } } } http://git-wip-us.apache.org/repos/asf/samza-hello-samza/blob/f25c37d3/src/main/java/samza/examples/cookbook/data/AdClick.java ---------------------------------------------------------------------- diff --git a/src/main/java/samza/examples/cookbook/data/AdClick.java b/src/main/java/samza/examples/cookbook/data/AdClick.java index 42d45dc..82925c7 100644 --- a/src/main/java/samza/examples/cookbook/data/AdClick.java +++ b/src/main/java/samza/examples/cookbook/data/AdClick.java @@ -19,6 +19,8 @@ package samza.examples.cookbook.data; +import org.codehaus.jackson.annotate.JsonProperty; + /** * An ad click event. */ @@ -28,6 +30,15 @@ public class AdClick { private String adId; // an unique id for the ad private String userId; // the user that clicked the ad + public AdClick( + @JsonProperty("pageId") String pageId, + @JsonProperty("adId") String adId, + @JsonProperty("userId") String userId) { + this.pageId = pageId; + this.adId = adId; + this.userId = userId; + } + public String getPageId() { return pageId; } http://git-wip-us.apache.org/repos/asf/samza-hello-samza/blob/f25c37d3/src/main/java/samza/examples/wikipedia/application/WikipediaApplication.java ---------------------------------------------------------------------- diff --git a/src/main/java/samza/examples/wikipedia/application/WikipediaApplication.java b/src/main/java/samza/examples/wikipedia/application/WikipediaApplication.java index 9077480..cda10df 100644 --- a/src/main/java/samza/examples/wikipedia/application/WikipediaApplication.java +++ b/src/main/java/samza/examples/wikipedia/application/WikipediaApplication.java @@ -21,6 +21,7 @@ package samza.examples.wikipedia.application; import com.google.common.collect.ImmutableList; import java.io.Serializable; +import java.util.Objects; import org.apache.samza.application.StreamApplication; import org.apache.samza.application.descriptors.StreamApplicationDescriptor; import org.apache.samza.context.Context; @@ -34,6 +35,8 @@ import org.apache.samza.operators.windows.Windows; import org.apache.samza.serializers.JsonSerdeV2; import org.apache.samza.serializers.Serde; import org.apache.samza.storage.kv.KeyValueStore; +import org.apache.samza.system.OutgoingMessageEnvelope; +import org.apache.samza.system.SystemStream; import org.apache.samza.system.kafka.descriptors.KafkaOutputDescriptor; import org.apache.samza.system.kafka.descriptors.KafkaSystemDescriptor; import org.slf4j.Logger; @@ -85,22 +88,28 @@ public class WikipediaApplication implements StreamApplication, Serializable { private static final List<String> KAFKA_PRODUCER_BOOTSTRAP_SERVERS = ImmutableList.of("localhost:9092"); private static final Map<String, String> KAFKA_DEFAULT_STREAM_CONFIGS = ImmutableMap.of("replication.factor", "1"); + public static final String WIKIPEDIA_CHANNEL = "#en.wikipedia"; + public static final String WIKINEWS_CHANNEL = "#en.wikinews"; + public static final String WIKTIONARY_CHANNEL = "#en.wiktionary"; + @Override public void describe(StreamApplicationDescriptor appDescriptor) { + Duration windowDuration = + appDescriptor.getConfig().containsKey("deploy.test") ? Duration.ofMillis(10) : Duration.ofSeconds(10); // Define a SystemDescriptor for Wikipedia data WikipediaSystemDescriptor wikipediaSystemDescriptor = new WikipediaSystemDescriptor("irc.wikimedia.org", 6667); // Define InputDescriptors for consuming wikipedia data WikipediaInputDescriptor wikipediaInputDescriptor = wikipediaSystemDescriptor .getInputDescriptor("en-wikipedia") - .withChannel("#en.wikipedia"); + .withChannel(WIKIPEDIA_CHANNEL); WikipediaInputDescriptor wiktionaryInputDescriptor = wikipediaSystemDescriptor .getInputDescriptor("en-wiktionary") - .withChannel("#en.wiktionary"); + .withChannel(WIKTIONARY_CHANNEL); WikipediaInputDescriptor wikiNewsInputDescriptor = wikipediaSystemDescriptor .getInputDescriptor("en-wikinews") - .withChannel("#en.wikinews"); + .withChannel(WIKINEWS_CHANNEL); // Define a system descriptor for Kafka KafkaSystemDescriptor kafkaSystemDescriptor = new KafkaSystemDescriptor("kafka") @@ -125,10 +134,11 @@ public class WikipediaApplication implements StreamApplication, Serializable { MessageStream<WikipediaFeedEvent> allWikipediaEvents = MessageStream.mergeAll(ImmutableList.of(wikipediaEvents, wiktionaryEvents, wikiNewsEvents)); + // Parse, update stats, prepare output, and send allWikipediaEvents .map(WikipediaParser::parseEvent) - .window(Windows.tumblingWindow(Duration.ofSeconds(10), + .window(Windows.tumblingWindow(windowDuration, WikipediaStats::new, new WikipediaStatsAggregator(), WikipediaStats.serde()), "statsWindow") .map(this::formatOutput) .sendTo(wikipediaStats); @@ -270,6 +280,26 @@ public class WikipediaApplication implements StreamApplication, Serializable { this.uniqueTitles = uniqueTitles; this.counts = counts; } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (o == null || getClass() != o.getClass()) { + return false; + } + WikipediaStatsOutput that = (WikipediaStatsOutput) o; + return edits == that.edits && editsAllTime == that.editsAllTime && bytesAdded == that.bytesAdded + && uniqueTitles == that.uniqueTitles && Objects.equals(counts, that.counts); + } + + @Override + public String toString() { + return "WikipediaStatsOutput{" + "edits=" + edits + ", editsAllTime=" + editsAllTime + ", bytesAdded=" + + bytesAdded + ", uniqueTitles=" + uniqueTitles + ", counts=" + counts + '}'; + } + } } http://git-wip-us.apache.org/repos/asf/samza-hello-samza/blob/f25c37d3/src/test/java/samza/examples/cookbook/test/TestSamzaCookBookExamples.java ---------------------------------------------------------------------- diff --git a/src/test/java/samza/examples/cookbook/test/TestSamzaCookBookExamples.java b/src/test/java/samza/examples/cookbook/test/TestSamzaCookBookExamples.java new file mode 100644 index 0000000..1e4b39a --- /dev/null +++ b/src/test/java/samza/examples/cookbook/test/TestSamzaCookBookExamples.java @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package samza.examples.cookbook.test; + +import java.time.Duration; +import java.util.ArrayList; +import java.util.List; +import org.apache.samza.operators.KV; +import org.apache.samza.serializers.NoOpSerde; +import org.apache.samza.test.framework.StreamAssert; +import org.apache.samza.test.framework.TestRunner; +import org.apache.samza.test.framework.system.descriptors.InMemoryInputDescriptor; +import org.apache.samza.test.framework.system.descriptors.InMemoryOutputDescriptor; +import org.apache.samza.test.framework.system.descriptors.InMemorySystemDescriptor; +import org.junit.Assert; +import org.junit.Test; +import samza.examples.cookbook.FilterExample; +import samza.examples.cookbook.JoinExample; +import samza.examples.cookbook.SessionWindowExample; +import samza.examples.cookbook.StreamTableJoinExample; +import samza.examples.cookbook.TumblingWindowExample; +import samza.examples.cookbook.data.AdClick; +import samza.examples.cookbook.data.PageView; +import samza.examples.cookbook.data.Profile; +import samza.examples.cookbook.data.UserPageViews; +import samza.examples.test.utils.TestUtils; + +import static samza.examples.cookbook.StreamTableJoinExample.EnrichedPageView; + + +public class TestSamzaCookBookExamples { + @Test + public void testFilterExample() { + List<PageView> rawPageViewEvents = new ArrayList<>(); + rawPageViewEvents.add(new PageView("google.com", "user1", "india")); + rawPageViewEvents.add(new PageView("facebook.com", "invalidUserId", "france")); + rawPageViewEvents.add(new PageView("yahoo.com", "user2", "china")); + + InMemorySystemDescriptor inMemorySystem = new InMemorySystemDescriptor("kafka"); + + InMemoryInputDescriptor<PageView> badPageViewEvents = + inMemorySystem.getInputDescriptor("pageview-filter-input", new NoOpSerde<PageView>()); + + InMemoryOutputDescriptor<PageView> goodPageViewEvents = + inMemorySystem.getOutputDescriptor("pageview-filter-output", new NoOpSerde<PageView>()); + + TestRunner + .of(new FilterExample()) + .addInputStream(badPageViewEvents, rawPageViewEvents) + .addOutputStream(goodPageViewEvents, 1) + .run(Duration.ofMillis(1500)); + + Assert.assertEquals(TestRunner.consumeStream(goodPageViewEvents, Duration.ofMillis(1000)).get(0).size(), 2); + } + + @Test + public void testJoinExample() { + List<PageView> pageViewEvents = new ArrayList<>(); + pageViewEvents.add(new PageView("google.com", "user1", "india")); + pageViewEvents.add(new PageView("yahoo.com", "user2", "china")); + List<AdClick> adClickEvents = new ArrayList<>(); + adClickEvents.add(new AdClick("google.com", "adClickId1", "user1")); + adClickEvents.add(new AdClick("yahoo.com", "adClickId2", "user1")); + + InMemorySystemDescriptor inMemorySystem = new InMemorySystemDescriptor("kafka"); + + InMemoryInputDescriptor<PageView> pageViews = + inMemorySystem.getInputDescriptor("pageview-join-input", new NoOpSerde<PageView>()); + + InMemoryInputDescriptor<AdClick> adClicks = + inMemorySystem.getInputDescriptor("adclick-join-input", new NoOpSerde<AdClick>()); + + InMemoryOutputDescriptor pageViewAdClickJoin = + inMemorySystem.getOutputDescriptor("pageview-adclick-join-output", new NoOpSerde<>()); + + TestRunner + .of(new JoinExample()) + .addInputStream(pageViews, pageViewEvents) + .addInputStream(adClicks, adClickEvents) + .addOutputStream(pageViewAdClickJoin, 1) + .run(Duration.ofMillis(1500)); + + Assert.assertEquals(TestRunner.consumeStream(pageViewAdClickJoin, Duration.ofMillis(1000)).get(0).size(), 2); + } + + @Test + public void testTumblingWindowExample() { + List<PageView> pageViewEvents = TestUtils.genSamplePageViewData(); + + InMemorySystemDescriptor inMemorySystem = new InMemorySystemDescriptor("kafka"); + + InMemoryInputDescriptor<KV<String, PageView>> pageViewInputDescriptor = + inMemorySystem.getInputDescriptor("pageview-tumbling-input", new NoOpSerde<KV<String, PageView>>()); + + InMemoryOutputDescriptor<KV<String, UserPageViews>> userPageViewOutputDescriptor = + inMemorySystem.getOutputDescriptor("pageview-tumbling-output", new NoOpSerde<KV<String, UserPageViews>>()); + + TestRunner + .of(new TumblingWindowExample()) + .addInputStream(pageViewInputDescriptor, pageViewEvents) + .addOutputStream(userPageViewOutputDescriptor, 1) + .run(Duration.ofMinutes(1)); + + Assert.assertTrue(TestRunner.consumeStream(userPageViewOutputDescriptor, Duration.ofMillis(1000)).get(0).size() > 1); + } + + @Test + public void testSessionWindowExample() { + List<PageView> pageViewEvents = TestUtils.genSamplePageViewData(); + + InMemorySystemDescriptor inMemorySystem = new InMemorySystemDescriptor("kafka"); + + InMemoryInputDescriptor<KV<String, PageView>> pageViewInputDescriptor = + inMemorySystem.getInputDescriptor("pageview-session-input", new NoOpSerde<KV<String, PageView>>()); + + InMemoryOutputDescriptor<KV<String, UserPageViews>> userPageViewOutputDescriptor = + inMemorySystem.getOutputDescriptor("pageview-session-output", new NoOpSerde<KV<String, UserPageViews>>()); + + TestRunner + .of(new SessionWindowExample()) + .addInputStream(pageViewInputDescriptor, pageViewEvents) + .addOutputStream(userPageViewOutputDescriptor, 1) + .run(Duration.ofMinutes(1)); + + Assert.assertEquals(2, TestRunner.consumeStream(userPageViewOutputDescriptor, Duration.ofMillis(1000)).get(0).size()); + } + + @Test + public void testStreamTableJoinExample() throws InterruptedException{ + List<PageView> pageViewEvents = new ArrayList<>(); + pageViewEvents.add(new PageView("google.com", "user1", "india")); + pageViewEvents.add(new PageView("yahoo.com", "user2", "china")); + List<Profile> profiles = new ArrayList<>(); + profiles.add(new Profile("user1", "LNKD")); + profiles.add(new Profile("user2", "MSFT")); + + InMemorySystemDescriptor inMemorySystem = new InMemorySystemDescriptor("kafka"); + + InMemoryInputDescriptor<PageView> pageViews = + inMemorySystem.getInputDescriptor("pageview-join-input", new NoOpSerde<PageView>()); + + InMemoryInputDescriptor<Profile> profileViews = + inMemorySystem.getInputDescriptor("profile-table-input", new NoOpSerde<Profile>()); + + InMemoryOutputDescriptor<EnrichedPageView> joinResultOutputDescriptor = + inMemorySystem.getOutputDescriptor("enriched-pageview-join-output", new NoOpSerde<EnrichedPageView>()); + + TestRunner + .of(new StreamTableJoinExample()) + .addInputStream(pageViews, pageViewEvents) + .addInputStream(profileViews, profiles) + .addOutputStream(joinResultOutputDescriptor, 1) + .run(Duration.ofMillis(1500)); + + List<EnrichedPageView> expectedOutput = new ArrayList<>(); + expectedOutput.add(new EnrichedPageView("user1", "LNKD", "google.com")); + expectedOutput.add(new EnrichedPageView("user2", "MSFT", "yahoo.com")); + + StreamAssert.containsInAnyOrder(expectedOutput, joinResultOutputDescriptor, Duration.ofMillis(200)); + + } + +} http://git-wip-us.apache.org/repos/asf/samza-hello-samza/blob/f25c37d3/src/test/java/samza/examples/test/utils/TestUtils.java ---------------------------------------------------------------------- diff --git a/src/test/java/samza/examples/test/utils/TestUtils.java b/src/test/java/samza/examples/test/utils/TestUtils.java new file mode 100644 index 0000000..d5e957e --- /dev/null +++ b/src/test/java/samza/examples/test/utils/TestUtils.java @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package samza.examples.test.utils; + +import com.google.common.io.Resources; +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.stream.Collectors; +import org.codehaus.jackson.map.ObjectMapper; +import samza.examples.cookbook.data.PageView; +import samza.examples.wikipedia.application.WikipediaApplication; + +import static samza.examples.wikipedia.system.WikipediaFeed.WikipediaFeedEvent; + + +public class TestUtils { + + public static List<WikipediaFeedEvent> genWikipediaFeedEvents(String channel) { + List<String> wikiEvents = null; + switch (channel) { + case WikipediaApplication.WIKIPEDIA_CHANNEL: + wikiEvents = readFile("WikipediaEditEvents.txt"); + break; + + case WikipediaApplication.WIKINEWS_CHANNEL: + wikiEvents = readFile("WikinewsEditEvents.txt"); + break; + + case WikipediaApplication.WIKTIONARY_CHANNEL: + wikiEvents = readFile("WikitionaryEditEvents.txt"); + break; + } + ObjectMapper mapper = new ObjectMapper(); + return wikiEvents.stream().map(event -> { + try { + return new WikipediaFeedEvent(mapper.readValue(event, HashMap.class)); + } catch (Exception e) { + e.printStackTrace(); + } + return null; + }).filter(x -> x != null).collect(Collectors.toList()); + } + + public static List<PageView> genSamplePageViewData() { + List<PageView> pageViewEvents = new ArrayList<>(); + pageViewEvents.add(new PageView("google.com/home", "user1", "india")); + pageViewEvents.add(new PageView("google.com/search", "user1", "india")); + pageViewEvents.add(new PageView("yahoo.com/home", "user2", "china")); + pageViewEvents.add(new PageView("yahoo.com/search", "user2", "china")); + pageViewEvents.add(new PageView("google.com/news", "user1", "india")); + pageViewEvents.add(new PageView("yahoo.com/fashion", "user2", "china")); + return pageViewEvents; + } + + private static List<String> readFile(String path) { + try { + InputStream in = Resources.getResource(path).openStream(); + List<String> lines = new ArrayList<>(); + String line = null; + BufferedReader reader = new BufferedReader(new InputStreamReader(in)); + while ((line = reader.readLine()) != null) { + lines.add(line); + } + reader.close(); + return lines; + } catch (IOException e) { + e.printStackTrace(); + return null; + } + } +} http://git-wip-us.apache.org/repos/asf/samza-hello-samza/blob/f25c37d3/src/test/java/samza/examples/wikipedia/application/test/TestWikipediaApplication.java ---------------------------------------------------------------------- diff --git a/src/test/java/samza/examples/wikipedia/application/test/TestWikipediaApplication.java b/src/test/java/samza/examples/wikipedia/application/test/TestWikipediaApplication.java new file mode 100644 index 0000000..dc1b5bd --- /dev/null +++ b/src/test/java/samza/examples/wikipedia/application/test/TestWikipediaApplication.java @@ -0,0 +1,82 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package samza.examples.wikipedia.application.test; + +import java.time.Duration; +import java.util.HashMap; +import java.util.Map; +import org.apache.samza.serializers.NoOpSerde; +import org.apache.samza.test.framework.TestRunner; +import org.apache.samza.test.framework.system.descriptors.InMemoryInputDescriptor; +import org.apache.samza.test.framework.system.descriptors.InMemoryOutputDescriptor; +import org.apache.samza.test.framework.system.descriptors.InMemorySystemDescriptor; +import org.junit.Assert; +import org.junit.Test; +import samza.examples.wikipedia.application.WikipediaApplication; +import samza.examples.test.utils.TestUtils; + + +public class TestWikipediaApplication { + + @Test + public void testWikipediaApplication() throws Exception { + + InMemorySystemDescriptor wikipediaSystemDescriptor = new InMemorySystemDescriptor("wikipedia"); + + // These config must be removed once examples are refactored to use Table-API + Map<String, String> conf = new HashMap<>(); + conf.put("stores.wikipedia-stats.factory", "org.apache.samza.storage.kv.RocksDbKeyValueStorageEngineFactory"); + conf.put("stores.wikipedia-stats.key.serde", "string"); + conf.put("stores.wikipedia-stats.msg.serde", "integer"); + conf.put("serializers.registry.string.class", "org.apache.samza.serializers.StringSerdeFactory"); + conf.put("serializers.registry.integer.class", "org.apache.samza.serializers.IntegerSerdeFactory"); + + InMemoryInputDescriptor wikipediaInputDescriptor = wikipediaSystemDescriptor + .getInputDescriptor("en-wikipedia", new NoOpSerde<>()) + .withPhysicalName(WikipediaApplication.WIKIPEDIA_CHANNEL); + + InMemoryInputDescriptor wiktionaryInputDescriptor = wikipediaSystemDescriptor + .getInputDescriptor("en-wiktionary", new NoOpSerde<>()) + .withPhysicalName(WikipediaApplication.WIKTIONARY_CHANNEL); + + InMemoryInputDescriptor wikiNewsInputDescriptor = wikipediaSystemDescriptor + .getInputDescriptor("en-wikinews", new NoOpSerde<>()) + .withPhysicalName(WikipediaApplication.WIKINEWS_CHANNEL); + + InMemorySystemDescriptor kafkaSystemDescriptor = new InMemorySystemDescriptor("kafka"); + + InMemoryOutputDescriptor outputStreamDesc = kafkaSystemDescriptor + .getOutputDescriptor("wikipedia-stats", new NoOpSerde<>()); + + + TestRunner + .of(new WikipediaApplication()) + .addInputStream(wikipediaInputDescriptor, TestUtils.genWikipediaFeedEvents(WikipediaApplication.WIKIPEDIA_CHANNEL)) + .addInputStream(wiktionaryInputDescriptor, TestUtils.genWikipediaFeedEvents(WikipediaApplication.WIKTIONARY_CHANNEL)) + .addInputStream(wikiNewsInputDescriptor, TestUtils.genWikipediaFeedEvents(WikipediaApplication.WIKINEWS_CHANNEL)) + .addOutputStream(outputStreamDesc, 1) + .addConfig(conf) + .addConfig("deploy.test", "true") + .run(Duration.ofMinutes(1)); + + Assert.assertTrue(TestRunner.consumeStream(outputStreamDesc, Duration.ofMillis(100)).get(0).size() > 0); + } + +} http://git-wip-us.apache.org/repos/asf/samza-hello-samza/blob/f25c37d3/src/test/java/samza/examples/wikipedia/task/test/TestWikipediaTask.java ---------------------------------------------------------------------- diff --git a/src/test/java/samza/examples/wikipedia/task/test/TestWikipediaTask.java b/src/test/java/samza/examples/wikipedia/task/test/TestWikipediaTask.java new file mode 100644 index 0000000..0fc992a --- /dev/null +++ b/src/test/java/samza/examples/wikipedia/task/test/TestWikipediaTask.java @@ -0,0 +1,70 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package samza.examples.wikipedia.task.test; + +import java.time.Duration; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import org.apache.samza.serializers.NoOpSerde; +import org.apache.samza.test.framework.TestRunner; +import org.apache.samza.test.framework.system.descriptors.InMemoryInputDescriptor; +import org.apache.samza.test.framework.system.descriptors.InMemoryOutputDescriptor; +import org.apache.samza.test.framework.system.descriptors.InMemorySystemDescriptor; +import org.codehaus.jackson.map.ObjectMapper; +import org.junit.Assert; +import org.junit.Test; +import samza.examples.wikipedia.system.WikipediaFeed.WikipediaFeedEvent; +import samza.examples.wikipedia.task.application.WikipediaParserTaskApplication; + +public class TestWikipediaTask { + + @Test + public void testWikipediaFeedTask() throws Exception { + String[] wikipediaFeedSamples = new String[] { "{\"channel\":\"#en.wikipedia\",\"raw\":\"[[Fear Is the Key (song)]] https://en.wikipedia.org/w/index.php?diff=865574761&oldid=861177329 * Sam Sailor * (+46) Redirecting to [[Fear of the Dark (Iron Maiden album)]] ([[User:Sam Sailor/Scripts/Sagittarius+|â]])\",\"time\":1540408899419,\"source\":\"rc-pmtpa\"}" }; + + InMemorySystemDescriptor isd = new InMemorySystemDescriptor("kafka"); + + InMemoryInputDescriptor rawWikiEvents = isd + .getInputDescriptor("wikipedia-raw", new NoOpSerde<>()); + + InMemoryOutputDescriptor<WikipediaFeedEvent> outputStreamDesc = isd + .getOutputDescriptor("wikipedia-edits", new NoOpSerde<>()); + + TestRunner + .of(new WikipediaParserTaskApplication()) + .addInputStream(rawWikiEvents, parseJSONToMap(wikipediaFeedSamples)) + .addOutputStream(outputStreamDesc, 1) + .run(Duration.ofSeconds(2)); + + Assert.assertEquals(1 + , TestRunner.consumeStream(outputStreamDesc, Duration.ofSeconds(1)).get(0).size()); + } + + public static List<Map<String, Object>> parseJSONToMap(String[] lines) throws Exception{ + List<Map<String, Object>> wikiRawEvents = new ArrayList<>(); + ObjectMapper mapper = new ObjectMapper(); + for (String line : lines) { + wikiRawEvents.add(mapper.readValue(line, HashMap.class)); + } + return wikiRawEvents; + } +} http://git-wip-us.apache.org/repos/asf/samza-hello-samza/blob/f25c37d3/src/test/resources/WikinewsEditEvents.txt ---------------------------------------------------------------------- diff --git a/src/test/resources/WikinewsEditEvents.txt b/src/test/resources/WikinewsEditEvents.txt new file mode 100644 index 0000000..2b192c3 --- /dev/null +++ b/src/test/resources/WikinewsEditEvents.txt @@ -0,0 +1,104 @@ +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Template:This Is Us]] !N https://en.wikinews.org/w/index.php?oldid=865574786&rcid=1098924965 * TheDoctorWho * (+497) Create","time":1540408910456,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[Downtown Houston]] https://en.wikinews.org/w/index.php?diff=865574783&oldid=865574723 * Doncram * (+78) /* See also */ [[National Register of Historic Places listings in downtown Houston, Texas]]","time":1540408907436,"source":"rc-pmtpa"} +{"channel":"#en.wikinews","raw":"[[List of countries by GNI (PPP) per capita]] https://en.wikinews.org/w/index.php?diff=865574785&oldid=865574706 * .24.196.115 * (+0) Undid revision 865491517 by [[Special:Contributions/Ducky mono is a beast killer|Ducky mono is a beast killer]] ([[User talk:Ducky mono is a beast killer|talk]])","time":1540408908317,"source":"rc-pmtpa"}