#18910: distributing descriptors accross CollecTor instances -------------------------------+--------------------------------- Reporter: iwakeh | Owner: iwakeh Type: enhancement | Status: needs_review Priority: High | Milestone: CollecTor 1.1.0 Component: Metrics/CollecTor | Version: Severity: Normal | Resolution: Keywords: ctip | Actual Points: Parent ID: | Points: Reviewer: | Sponsor: -------------------------------+---------------------------------
Comment (by karsten): Please take a look at the new commits in my updated task-18910-4 branch. Remaining tasks: - Find out why unit tests fail. (See below.) - Test on a server. - Squash commits and merge. Here's the test output with 4 failing tests, both on OS X and on Linux: {{{ [junit] Testsuite: org.torproject.collector.sync.SyncPersistenceTest [junit] Tests run: 40, Failures: 4, Errors: 0, Skipped: 0, Time elapsed: 30.708 sec [junit] [junit] Testcase: testDescWriteOutput[0] took 2.044 sec [junit] Testcase: testDescWriteRecent[0] took 0.523 sec [junit] Testcase: testOutFileContent[0] took 0.396 sec [junit] Testcase: testRecentFileContent[0] took 0.389 sec [junit] Testcase: testDescWriteOutput[1] took 0.454 sec [junit] FAILED [junit] data used: relay-descriptors/server- descriptor/2016/10/e/3/e381ce74a1a592f6d375706665aba6d4d22923f1, relay-2016-10-02-16-05-00-server-descriptors, resulting list: [/tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/c/8/c8c3588019f7c896eb4185cfc1074cfe5eb405ea, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/4/1/4179c50d3c764bc85c9d719e14e55a6cc232a10d, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/2/0/2091f76a8256e479cbe4f57be85f87909af07236, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/b/b/bbca7ed70ba6ea88f995b067a004f5a4d0903d6e, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/d/a/dae8966ca600b46bc75ed5efb97286481e9a6876, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/e/3/e381ce74a1a592f6d375706665aba6d4d22923f1, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/e/1/e1142337dee5b890393a0891acbde51577c2b743, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/a/0/a0ed9227a9413f140445002ce412f8828591e7ec, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/5/b/5b202650802a916f1ec3a1ef36b98706e3747701, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/5/a/5a536243bf056cd7177ddfd8eb363fec978f3343] expected:<...-descriptor/2016/10/[e/3/e381ce74a1a592f6d375706665aba6d4d22923f1]> but was:<...-descriptor/2016/10/[c/8/c8c3588019f7c896eb4185cfc1074cfe5eb405ea]> [junit] junit.framework.AssertionFailedError: data used: relay- descriptors/server- descriptor/2016/10/e/3/e381ce74a1a592f6d375706665aba6d4d22923f1, relay-2016-10-02-16-05-00-server-descriptors, resulting list: [/tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/c/8/c8c3588019f7c896eb4185cfc1074cfe5eb405ea, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/4/1/4179c50d3c764bc85c9d719e14e55a6cc232a10d, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/2/0/2091f76a8256e479cbe4f57be85f87909af07236, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/b/b/bbca7ed70ba6ea88f995b067a004f5a4d0903d6e, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/d/a/dae8966ca600b46bc75ed5efb97286481e9a6876, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/e/3/e381ce74a1a592f6d375706665aba6d4d22923f1, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/e/1/e1142337dee5b890393a0891acbde51577c2b743, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/a/0/a0ed9227a9413f140445002ce412f8828591e7ec, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/5/b/5b202650802a916f1ec3a1ef36b98706e3747701, /tmp/junit1420314177894412746/out/relay-descriptors/server- descriptor/2016/10/5/a/5a536243bf056cd7177ddfd8eb363fec978f3343] expected:<...-descriptor/2016/10/[e/3/e381ce74a1a592f6d375706665aba6d4d22923f1]> but was:<...-descriptor/2016/10/[c/8/c8c3588019f7c896eb4185cfc1074cfe5eb405ea]> [junit] at org.torproject.collector.sync.SyncPersistenceTest.testDescWriteOutput(SyncPersistenceTest.java:179) [junit] [junit] Testcase: testDescWriteRecent[1] took 0.057 sec [junit] Testcase: testOutFileContent[1] took 0.1 sec [junit] Testcase: testRecentFileContent[1] took 0.058 sec [junit] Testcase: testDescWriteOutput[2] took 1.398 sec [junit] Testcase: testDescWriteRecent[2] took 0.505 sec [junit] Testcase: testOutFileContent[2] took 0.858 sec [junit] Testcase: testRecentFileContent[2] took 0.722 sec [junit] Testcase: testDescWriteOutput[3] took 0.434 sec [junit] Testcase: testDescWriteRecent[3] took 0.43 sec [junit] Testcase: testOutFileContent[3] took 0.673 sec [junit] Testcase: testRecentFileContent[3] took 0.67 sec [junit] Testcase: testDescWriteOutput[4] took 1.336 sec [junit] Testcase: testDescWriteRecent[4] took 0.861 sec [junit] Testcase: testOutFileContent[4] took 1.484 sec [junit] Testcase: testRecentFileContent[4] took 1.431 sec [junit] Testcase: testDescWriteOutput[5] took 0.867 sec [junit] Testcase: testDescWriteRecent[5] took 0.8 sec [junit] Testcase: testOutFileContent[5] took 1.601 sec [junit] Testcase: testRecentFileContent[5] took 1.7 sec [junit] Testcase: testDescWriteOutput[6] took 0.19 sec [junit] FAILED [junit] data used: relay-descriptors/extra- info/2016/10/9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c, relay-2016-10-02-08-05-00-extra-infos, resulting list: [/tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/c/a/ca86eb96d22d188bb574b6b329ab21e0d9243516, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/3/1/317586098443ed19b200417556a08ebc42133521, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/3/6/36691feb7cec6a9630b9ecd11a9b5dc61c147c5d, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/4/e/4ef90738e54a403b265120dcbab7b494e0c68d3b, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/8/2/82471deac7b251089a0878d29a228d4e323b823f, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/0/4/04219ada0be922fa7518d36b0d8e66afc55e8603, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/e/b/eb73b59951bc1b0403be81220fb75be464954c31, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/6/a/6a36d4ac36447e645c91ed63633a09197b7ad97e] expected :<.../extra-info/2016/10/[9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c]> but was:<.../extra- info/2016/10/[c/a/ca86eb96d22d188bb574b6b329ab21e0d9243516]> [junit] junit.framework.AssertionFailedError: data used: relay- descriptors/extra- info/2016/10/9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c, relay-2016-10-02-08-05-00-extra-infos, resulting list: [/tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/c/a/ca86eb96d22d188bb574b6b329ab21e0d9243516, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/3/1/317586098443ed19b200417556a08ebc42133521, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/3/6/36691feb7cec6a9630b9ecd11a9b5dc61c147c5d, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/4/e/4ef90738e54a403b265120dcbab7b494e0c68d3b, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/8/2/82471deac7b251089a0878d29a228d4e323b823f, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/0/4/04219ada0be922fa7518d36b0d8e66afc55e8603, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/e/b/eb73b59951bc1b0403be81220fb75be464954c31, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c, /tmp/junit4173269913956546333/out/relay-descriptors/extra- info/2016/10/6/a/6a36d4ac36447e645c91ed63633a09197b7ad97e] expected :<.../extra-info/2016/10/[9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c]> but was:<.../extra- info/2016/10/[c/a/ca86eb96d22d188bb574b6b329ab21e0d9243516]> [junit] at org.torproject.collector.sync.SyncPersistenceTest.testDescWriteOutput(SyncPersistenceTest.java:179) [junit] [junit] Testcase: testDescWriteRecent[6] took 0.038 sec [junit] Testcase: testOutFileContent[6] took 0.048 sec [junit] Testcase: testRecentFileContent[6] took 0.092 sec [junit] Testcase: testDescWriteOutput[7] took 0.086 sec [junit] FAILED [junit] data used: bridge-descriptors/2016/10/extra- infos/9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829, bridge-2016-10-02-08-09-00-extra-infos, resulting list: [/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/c/a/cab78ea0ffe9a7bc00527fef19f546c47d59f01a, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/3/4/3412a1dccd183a1c0bd1b748f34d88594be6ea52, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/3/1/31bcea576e77ba66150f7903b588c919adad849c, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/1/6/16d2b79fbd0a8567c6afd7585c775ac7745561e3, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/f/a/fabd8f614633ec2d2d405f2554e14381bc33d9cb, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/b/c/bcfcbb38b15e9b500b1a6e9b0bcbbce858660f17, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/e/1/e11c5239494bad2f6f3759f1104a2f6182beab4d, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/e/e/eee0dc51b9a0a71ba73610123b13cea212b5cf83, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/7/a/7a93ca1edc543e747f1157bc3a557890335311a4] expected:<...2016/10 /extra-infos/[9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829]> but was:<...2016/10/extra- infos/[c/a/cab78ea0ffe9a7bc00527fef19f546c47d59f01a]> [junit] junit.framework.AssertionFailedError: data used: bridge- descriptors/2016/10/extra- infos/9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829, bridge-2016-10-02-08-09-00-extra-infos, resulting list: [/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/c/a/cab78ea0ffe9a7bc00527fef19f546c47d59f01a, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/3/4/3412a1dccd183a1c0bd1b748f34d88594be6ea52, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/3/1/31bcea576e77ba66150f7903b588c919adad849c, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/1/6/16d2b79fbd0a8567c6afd7585c775ac7745561e3, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/f/a/fabd8f614633ec2d2d405f2554e14381bc33d9cb, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/b/c/bcfcbb38b15e9b500b1a6e9b0bcbbce858660f17, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/e/1/e11c5239494bad2f6f3759f1104a2f6182beab4d, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/e/e/eee0dc51b9a0a71ba73610123b13cea212b5cf83, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829, /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra- infos/7/a/7a93ca1edc543e747f1157bc3a557890335311a4] expected:<...2016/10 /extra-infos/[9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829]> but was:<...2016/10/extra- infos/[c/a/cab78ea0ffe9a7bc00527fef19f546c47d59f01a]> [junit] at org.torproject.collector.sync.SyncPersistenceTest.testDescWriteOutput(SyncPersistenceTest.java:179) [junit] [junit] Testcase: testDescWriteRecent[7] took 0.027 sec [junit] Testcase: testOutFileContent[7] took 0.071 sec [junit] Testcase: testRecentFileContent[7] took 0.054 sec [junit] Testcase: testDescWriteOutput[8] took 0.07 sec [junit] FAILED [junit] data used: bridge-descriptors/2016/10/server- descriptors/a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872, bridge-2016-10-02-16-09-00-server-descriptors, resulting list: [/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/c/3/c32acc52826b37e5319f1bac2f8812b33a772540, /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/0/7/07d952e9020cb68a63d9156653a2e41af4af4d44, /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/0/a/0a65c636a20631bd5deb2f10dc664d2032303c46, /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/e/5/e5d46e88cb52d4fc0524398cfb7a6754394bc5e9, /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/6/1/614414898ee133ee9bf6b10a9898cab518c5453e, /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/7/8/784d6f9e82426072fbfc7a42f8f7133ba6ec1453, /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872, /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/5/b/5b20316b03afc98a165a219044b6fa6ca34c58ab] expected :<.../server-descriptors/[a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872]> but was:<.../server- descriptors/[c/3/c32acc52826b37e5319f1bac2f8812b33a772540]> [junit] junit.framework.AssertionFailedError: data used: bridge- descriptors/2016/10/server- descriptors/a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872, bridge-2016-10-02-16-09-00-server-descriptors, resulting list: [/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/c/3/c32acc52826b37e5319f1bac2f8812b33a772540, /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/0/7/07d952e9020cb68a63d9156653a2e41af4af4d44, /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/0/a/0a65c636a20631bd5deb2f10dc664d2032303c46, /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/e/5/e5d46e88cb52d4fc0524398cfb7a6754394bc5e9, /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/6/1/614414898ee133ee9bf6b10a9898cab518c5453e, /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/7/8/784d6f9e82426072fbfc7a42f8f7133ba6ec1453, /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872, /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server- descriptors/5/b/5b20316b03afc98a165a219044b6fa6ca34c58ab] expected :<.../server-descriptors/[a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872]> but was:<.../server- descriptors/[c/3/c32acc52826b37e5319f1bac2f8812b33a772540]> [junit] at org.torproject.collector.sync.SyncPersistenceTest.testDescWriteOutput(SyncPersistenceTest.java:179) [junit] [junit] Testcase: testDescWriteRecent[8] took 0.025 sec [junit] Testcase: testOutFileContent[8] took 0.036 sec [junit] Testcase: testRecentFileContent[8] took 0.036 sec [junit] Testcase: testDescWriteOutput[9] took 0.192 sec [junit] Testcase: testDescWriteRecent[9] took 0.086 sec [junit] Testcase: testOutFileContent[9] took 0.1 sec [junit] Testcase: testRecentFileContent[9] took 0.109 sec }}} -- Ticket URL: <https://trac.torproject.org/projects/tor/ticket/18910#comment:67> Tor Bug Tracker & Wiki <https://trac.torproject.org/> The Tor Project: anonymity online _______________________________________________ tor-bugs mailing list tor-bugs@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs