[Wikidata-bugs] [Maniphest] [Commented On] T178530: Improve field mapping for nginx logstash
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2017-10-25T14:52:33Z] applying new template to elasticsearch / logstash - T178530TASK DETAILhttps://phabricator.wikimedia.org/T178530EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: StashbotCc: Stashbot, gerritbot, dcausse, Gehel, EBernhardson, Aklapper, Smalyshev, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, Avner, Lewizho99, Maathavan, debt, Jdrewniak, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331, fgiunchedi, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T178530: Improve field mapping for nginx logstash
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2017-10-25T14:51:30Z] update logstash template on logstash elsaticsearch cluster for T178530TASK DETAILhttps://phabricator.wikimedia.org/T178530EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: StashbotCc: Stashbot, gerritbot, dcausse, Gehel, EBernhardson, Aklapper, Smalyshev, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, Avner, Lewizho99, Maathavan, debt, Jdrewniak, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331, fgiunchedi, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T178530: Improve field mapping for nginx logstash
Gehel added a comment. Change merged and deployed on all logstash servers. If I understand correctly, we need to wait for new index creation to check if this worked correctly.TASK DETAILhttps://phabricator.wikimedia.org/T178530EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: GehelCc: gerritbot, dcausse, Gehel, EBernhardson, Aklapper, Smalyshev, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, Avner, Lewizho99, Maathavan, debt, Jdrewniak, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331, fgiunchedi, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T178530: Improve field mapping for nginx logstash
gerritbot added a comment. Change 386317 merged by Gehel: [operations/puppet@production] Add types to some fields used by nginx https://gerrit.wikimedia.org/r/386317TASK DETAILhttps://phabricator.wikimedia.org/T178530EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: gerritbot, dcausse, Gehel, EBernhardson, Aklapper, Smalyshev, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, Avner, Lewizho99, Maathavan, debt, Jdrewniak, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331, fgiunchedi, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T178530: Improve field mapping for nginx logstash
gerritbot added a comment. Change 386317 had a related patch set uploaded (by Smalyshev; owner: Smalyshev): [operations/puppet@production] Add types to some fields used by nginx https://gerrit.wikimedia.org/r/386317TASK DETAILhttps://phabricator.wikimedia.org/T178530EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: gerritbot, dcausse, Gehel, EBernhardson, Aklapper, Smalyshev, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, Avner, debt, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331, fgiunchedi, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T178530: Improve field mapping for nginx logstash
EBernhardson added a comment. There might be other opinions, but i think hard coding specific fields to specific types in the logstash config is reasonable as long as it's documented. The primary problem we run into is that without coordination different types can be sent to fields by different applications. While documenting and expecting applications to conform to types for all parameters is not going to happen, doing it for some limited set of useful fields seems acceptable to me.TASK DETAILhttps://phabricator.wikimedia.org/T178530EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: EBernhardsonCc: dcausse, Gehel, EBernhardson, Aklapper, Smalyshev, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, Avner, debt, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T178530: Improve field mapping for nginx logstash
Smalyshev added a comment. My checks show that request_time and upstream_time only used by nginx, but response_size is also used by webrequest. While it is used in the same meaning, I wonder if changing the type won't disrupt anything. response_size is less important for me than the rest, so if it proves a problem, we could omit it.TASK DETAILhttps://phabricator.wikimedia.org/T178530EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: dcausse, Gehel, EBernhardson, Aklapper, Smalyshev, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, Avner, debt, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
