Re: [Analytics] EventLogging Hive Refine currently stalled for some Schemas
Following up on this, data is been re-ingested into druid so turnilo should not show up any holes related to this event. Thanks, Nuria On Thu, Nov 15, 2018 at 8:45 AM Marcel Ruiz Forns wrote: > Not all data sources are populated at the same time, the data on Druid is >> ingested twice, once per hour and once daily looking 4 days back. Data >> should appear once daily job runs for the "holes" missing. > > +1 The EL2Druid daily loading job will cover up the holes for the 12th > and 13th in 1 or 2 days. > > On Thu, Nov 15, 2018 at 5:03 PM Nuria Ruiz wrote: > >> Hello, >> >> Not all data sources are populated at the same time, the data on Druid is >> ingested twice, once per hour and once daily looking 4 days back. Data >> should appear once daily job runs for the "holes" missing. >> >> Thanks, >> >> Nuria >> >> On Thu, Nov 15, 2018 at 7:49 AM Andrew Otto wrote: >> >>> > Does "fixed" mean that the missing data already been backfilled? I’m >>> seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the >>> timespans between around 6am-16pm on November 13, and 7am-10am on November >>> 12. >>> >>> Hm. Fixed means the data has been refined to Hive. I didn’t check on >>> Druid EL data. Marcel, can you check up on that? This sounds related, we >>> might need to figure out how to re run the Druid ingestion if/when base >>> data partitions change. >>> >>> >>> >>> On Thu, Nov 15, 2018 at 10:17 AM Tilman Bayer >>> wrote: >>> Hi Andrew, PS, offtopic: reminder that data-analy...@wikimedia.org has been deprecated/retired a while ago; our team can be reached at product-analyt...@wikimedia.org (CCed) On Thu, Nov 15, 2018 at 7:13 AM Tilman Bayer wrote: > Does "fixed" mean that the missing data already been backfilled? I'm > seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the > timespans between around 6am-16pm on November 13, and 7am-10am on November > 12. > > On Thu, Nov 15, 2018 at 6:51 AM Andrew Otto > wrote: > >> OH I’m sorry! There is a Phab task, and this is fixed. >> https://phabricator.wikimedia.org/T209407 >> >> Very sorry, I should have updated and linked the task. >> >> On Thu, Nov 15, 2018 at 9:50 AM Gilles Dubuc >> wrote: >> >>> Is this issue still ongoing? Is there a corresponding Phabricator >>> task to follow? >>> >>> On Tue, Nov 13, 2018 at 6:27 PM Andrew Otto >>> wrote: >>> Hi all, Yesterday we upgraded the Hadoop cluster to a newer version. It seems that along the way the job that imports EventLogging data into Hive has started failing for some EventLogging datasets. I’m still investigating, but it seems that tables (I believe ones that need to have their schema modified during refinement) are failing refinement. The list of affected tables at this moment is large, and will likely only grow as time goes on. We’re looking into this and will try to have things fixed ASAP. For now you might expect Hive EventLogging data to be delayed. Sorry for the trouble, and we’ll keep you updated! -Andrew Otto Systems Engineer, WMF The list of currently affected tables is: AdvancedSearchRequest CentralAuth CentralNoticeBannerHistory CentralNoticeImpression CentralNoticeTiming ChangesListFilterGrouping ChangesListFilters CitationUsage CitationUsagePageLoad ContentTranslation ContentTranslationAbuseFilter ContentTranslationCTA ContentTranslationError ContentTranslationSuggestion CpuBenchmark EchoInteraction EchoMail EditAttemptStep EditConflict EditorActivation EUCCVisit EventError FlowReplies GettingStartedRedirectImpression GuidedTourButtonClick GuidedTourExited GuidedTourExternalLinkActivation GuidedTourGuiderHidden GuidedTourGuiderImpression LandingPageImpression MediaViewer MediaWikiPingback MobileAppCategorizationAttempts MobileAppUploadAttempts MobileWebMainMenuClickTracking MobileWebSearch MobileWebUIClickTracking MobileWikiAppAppearanceSettings MobileWikiAppArticleSuggestions MobileWikiAppCreateAccount MobileWikiAppDailyStats MobileWikiAppEdit MobileWikiAppFeed MobileWikiAppFeedConfigure MobileWikiAppFindInPage MobileWikiAppInstallReferrer MobileWikiAppIntents MobileWikiAppiOSFeed MobileWikiAppiOSLoginAction MobileWikiAppiOSReadingLists
Re: [Analytics] EventLogging Hive Refine currently stalled for some Schemas
> > Not all data sources are populated at the same time, the data on Druid is > ingested twice, once per hour and once daily looking 4 days back. Data > should appear once daily job runs for the "holes" missing. +1 The EL2Druid daily loading job will cover up the holes for the 12th and 13th in 1 or 2 days. On Thu, Nov 15, 2018 at 5:03 PM Nuria Ruiz wrote: > Hello, > > Not all data sources are populated at the same time, the data on Druid is > ingested twice, once per hour and once daily looking 4 days back. Data > should appear once daily job runs for the "holes" missing. > > Thanks, > > Nuria > > On Thu, Nov 15, 2018 at 7:49 AM Andrew Otto wrote: > >> > Does "fixed" mean that the missing data already been backfilled? I’m >> seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the >> timespans between around 6am-16pm on November 13, and 7am-10am on November >> 12. >> >> Hm. Fixed means the data has been refined to Hive. I didn’t check on >> Druid EL data. Marcel, can you check up on that? This sounds related, we >> might need to figure out how to re run the Druid ingestion if/when base >> data partitions change. >> >> >> >> On Thu, Nov 15, 2018 at 10:17 AM Tilman Bayer >> wrote: >> >>> Hi Andrew, >>> >>> PS, offtopic: reminder that data-analy...@wikimedia.org has been >>> deprecated/retired a while ago; our team can be reached at >>> product-analyt...@wikimedia.org (CCed) >>> >>> On Thu, Nov 15, 2018 at 7:13 AM Tilman Bayer >>> wrote: >>> Does "fixed" mean that the missing data already been backfilled? I'm seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the timespans between around 6am-16pm on November 13, and 7am-10am on November 12. On Thu, Nov 15, 2018 at 6:51 AM Andrew Otto wrote: > OH I’m sorry! There is a Phab task, and this is fixed. > https://phabricator.wikimedia.org/T209407 > > Very sorry, I should have updated and linked the task. > > On Thu, Nov 15, 2018 at 9:50 AM Gilles Dubuc > wrote: > >> Is this issue still ongoing? Is there a corresponding Phabricator >> task to follow? >> >> On Tue, Nov 13, 2018 at 6:27 PM Andrew Otto >> wrote: >> >>> Hi all, >>> >>> Yesterday we upgraded the Hadoop cluster to a newer version. It >>> seems that along the way the job that imports EventLogging data into >>> Hive >>> has started failing for some EventLogging datasets. I’m still >>> investigating, but it seems that tables (I believe ones that need to >>> have >>> their schema modified during refinement) are failing refinement. The >>> list >>> of affected tables at this moment is large, and will likely only grow as >>> time goes on. >>> >>> We’re looking into this and will try to have things fixed ASAP. For >>> now you might expect Hive EventLogging data to be delayed. >>> >>> Sorry for the trouble, and we’ll keep you updated! >>> -Andrew Otto >>> Systems Engineer, WMF >>> >>> >>> >>> The list of currently affected tables is: >>> >>> AdvancedSearchRequest >>> CentralAuth >>> CentralNoticeBannerHistory >>> CentralNoticeImpression >>> CentralNoticeTiming >>> ChangesListFilterGrouping >>> ChangesListFilters >>> CitationUsage >>> CitationUsagePageLoad >>> ContentTranslation >>> ContentTranslationAbuseFilter >>> ContentTranslationCTA >>> ContentTranslationError >>> ContentTranslationSuggestion >>> CpuBenchmark >>> EchoInteraction >>> EchoMail >>> EditAttemptStep >>> EditConflict >>> EditorActivation >>> EUCCVisit >>> EventError >>> FlowReplies >>> GettingStartedRedirectImpression >>> GuidedTourButtonClick >>> GuidedTourExited >>> GuidedTourExternalLinkActivation >>> GuidedTourGuiderHidden >>> GuidedTourGuiderImpression >>> LandingPageImpression >>> MediaViewer >>> MediaWikiPingback >>> MobileAppCategorizationAttempts >>> MobileAppUploadAttempts >>> MobileWebMainMenuClickTracking >>> MobileWebSearch >>> MobileWebUIClickTracking >>> MobileWikiAppAppearanceSettings >>> MobileWikiAppArticleSuggestions >>> MobileWikiAppCreateAccount >>> MobileWikiAppDailyStats >>> MobileWikiAppEdit >>> MobileWikiAppFeed >>> MobileWikiAppFeedConfigure >>> MobileWikiAppFindInPage >>> MobileWikiAppInstallReferrer >>> MobileWikiAppIntents >>> MobileWikiAppiOSFeed >>> MobileWikiAppiOSLoginAction >>> MobileWikiAppiOSReadingLists >>> MobileWikiAppiOSSearch >>> MobileWikiAppiOSSessions >>> MobileWikiAppiOSSettingAction >>> MobileWikiAppiOSUserHistory >>> MobileWikiAppLangSelect >>> MobileWikiAppLanguageSearching >>> MobileWikiAppLanguageSettings >>> MobileWikiAppLinkPreview >>> MobileWikiAppLogin >>> MobileWikiAppMediaGallery >>> MobileWikiAppNavMenu >>>
Re: [Analytics] EventLogging Hive Refine currently stalled for some Schemas
Hello, Not all data sources are populated at the same time, the data on Druid is ingested twice, once per hour and once daily looking 4 days back. Data should appear once daily job runs for the "holes" missing. Thanks, Nuria On Thu, Nov 15, 2018 at 7:49 AM Andrew Otto wrote: > > Does "fixed" mean that the missing data already been backfilled? I’m > seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the > timespans between around 6am-16pm on November 13, and 7am-10am on November > 12. > > Hm. Fixed means the data has been refined to Hive. I didn’t check on > Druid EL data. Marcel, can you check up on that? This sounds related, we > might need to figure out how to re run the Druid ingestion if/when base > data partitions change. > > > > On Thu, Nov 15, 2018 at 10:17 AM Tilman Bayer > wrote: > >> Hi Andrew, >> >> PS, offtopic: reminder that data-analy...@wikimedia.org has been >> deprecated/retired a while ago; our team can be reached at >> product-analyt...@wikimedia.org (CCed) >> >> On Thu, Nov 15, 2018 at 7:13 AM Tilman Bayer >> wrote: >> >>> Does "fixed" mean that the missing data already been backfilled? I'm >>> seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the >>> timespans between around 6am-16pm on November 13, and 7am-10am on November >>> 12. >>> >>> On Thu, Nov 15, 2018 at 6:51 AM Andrew Otto wrote: >>> OH I’m sorry! There is a Phab task, and this is fixed. https://phabricator.wikimedia.org/T209407 Very sorry, I should have updated and linked the task. On Thu, Nov 15, 2018 at 9:50 AM Gilles Dubuc wrote: > Is this issue still ongoing? Is there a corresponding Phabricator task > to follow? > > On Tue, Nov 13, 2018 at 6:27 PM Andrew Otto > wrote: > >> Hi all, >> >> Yesterday we upgraded the Hadoop cluster to a newer version. It >> seems that along the way the job that imports EventLogging data into Hive >> has started failing for some EventLogging datasets. I’m still >> investigating, but it seems that tables (I believe ones that need to have >> their schema modified during refinement) are failing refinement. The >> list >> of affected tables at this moment is large, and will likely only grow as >> time goes on. >> >> We’re looking into this and will try to have things fixed ASAP. For >> now you might expect Hive EventLogging data to be delayed. >> >> Sorry for the trouble, and we’ll keep you updated! >> -Andrew Otto >> Systems Engineer, WMF >> >> >> >> The list of currently affected tables is: >> >> AdvancedSearchRequest >> CentralAuth >> CentralNoticeBannerHistory >> CentralNoticeImpression >> CentralNoticeTiming >> ChangesListFilterGrouping >> ChangesListFilters >> CitationUsage >> CitationUsagePageLoad >> ContentTranslation >> ContentTranslationAbuseFilter >> ContentTranslationCTA >> ContentTranslationError >> ContentTranslationSuggestion >> CpuBenchmark >> EchoInteraction >> EchoMail >> EditAttemptStep >> EditConflict >> EditorActivation >> EUCCVisit >> EventError >> FlowReplies >> GettingStartedRedirectImpression >> GuidedTourButtonClick >> GuidedTourExited >> GuidedTourExternalLinkActivation >> GuidedTourGuiderHidden >> GuidedTourGuiderImpression >> LandingPageImpression >> MediaViewer >> MediaWikiPingback >> MobileAppCategorizationAttempts >> MobileAppUploadAttempts >> MobileWebMainMenuClickTracking >> MobileWebSearch >> MobileWebUIClickTracking >> MobileWikiAppAppearanceSettings >> MobileWikiAppArticleSuggestions >> MobileWikiAppCreateAccount >> MobileWikiAppDailyStats >> MobileWikiAppEdit >> MobileWikiAppFeed >> MobileWikiAppFeedConfigure >> MobileWikiAppFindInPage >> MobileWikiAppInstallReferrer >> MobileWikiAppIntents >> MobileWikiAppiOSFeed >> MobileWikiAppiOSLoginAction >> MobileWikiAppiOSReadingLists >> MobileWikiAppiOSSearch >> MobileWikiAppiOSSessions >> MobileWikiAppiOSSettingAction >> MobileWikiAppiOSUserHistory >> MobileWikiAppLangSelect >> MobileWikiAppLanguageSearching >> MobileWikiAppLanguageSettings >> MobileWikiAppLinkPreview >> MobileWikiAppLogin >> MobileWikiAppMediaGallery >> MobileWikiAppNavMenu >> MobileWikiAppNotificationInteraction >> MobileWikiAppNotificationPreferences >> MobileWikiAppOfflineLibrary >> MobileWikiAppOnboarding >> MobileWikiAppOnThisDay >> MobileWikiAppPageScroll >> MobileWikiAppProtectedEditAttempt >> MobileWikiAppRandomizer >> MobileWikiAppReadingLists >> MobileWikiAppSavedPages >> MobileWikiAppSearch >> MobileWikiAppSessions >> MobileWikiAppShareAFact >> MobileWikiAppStuffHappens >> MobileWikiAppTabs >> MobileWikiAppToCInteraction >>
Re: [Analytics] EventLogging Hive Refine currently stalled for some Schemas
> Does "fixed" mean that the missing data already been backfilled? I’m seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the timespans between around 6am-16pm on November 13, and 7am-10am on November 12. Hm. Fixed means the data has been refined to Hive. I didn’t check on Druid EL data. Marcel, can you check up on that? This sounds related, we might need to figure out how to re run the Druid ingestion if/when base data partitions change. On Thu, Nov 15, 2018 at 10:17 AM Tilman Bayer wrote: > Hi Andrew, > > PS, offtopic: reminder that data-analy...@wikimedia.org has been > deprecated/retired a while ago; our team can be reached at > product-analyt...@wikimedia.org (CCed) > > On Thu, Nov 15, 2018 at 7:13 AM Tilman Bayer wrote: > >> Does "fixed" mean that the missing data already been backfilled? I'm >> seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the >> timespans between around 6am-16pm on November 13, and 7am-10am on November >> 12. >> >> On Thu, Nov 15, 2018 at 6:51 AM Andrew Otto wrote: >> >>> OH I’m sorry! There is a Phab task, and this is fixed. >>> https://phabricator.wikimedia.org/T209407 >>> >>> Very sorry, I should have updated and linked the task. >>> >>> On Thu, Nov 15, 2018 at 9:50 AM Gilles Dubuc >>> wrote: >>> Is this issue still ongoing? Is there a corresponding Phabricator task to follow? On Tue, Nov 13, 2018 at 6:27 PM Andrew Otto wrote: > Hi all, > > Yesterday we upgraded the Hadoop cluster to a newer version. It seems > that along the way the job that imports EventLogging data into Hive has > started failing for some EventLogging datasets. I’m still investigating, > but it seems that tables (I believe ones that need to have their schema > modified during refinement) are failing refinement. The list of affected > tables at this moment is large, and will likely only grow as time goes on. > > We’re looking into this and will try to have things fixed ASAP. For > now you might expect Hive EventLogging data to be delayed. > > Sorry for the trouble, and we’ll keep you updated! > -Andrew Otto > Systems Engineer, WMF > > > > The list of currently affected tables is: > > AdvancedSearchRequest > CentralAuth > CentralNoticeBannerHistory > CentralNoticeImpression > CentralNoticeTiming > ChangesListFilterGrouping > ChangesListFilters > CitationUsage > CitationUsagePageLoad > ContentTranslation > ContentTranslationAbuseFilter > ContentTranslationCTA > ContentTranslationError > ContentTranslationSuggestion > CpuBenchmark > EchoInteraction > EchoMail > EditAttemptStep > EditConflict > EditorActivation > EUCCVisit > EventError > FlowReplies > GettingStartedRedirectImpression > GuidedTourButtonClick > GuidedTourExited > GuidedTourExternalLinkActivation > GuidedTourGuiderHidden > GuidedTourGuiderImpression > LandingPageImpression > MediaViewer > MediaWikiPingback > MobileAppCategorizationAttempts > MobileAppUploadAttempts > MobileWebMainMenuClickTracking > MobileWebSearch > MobileWebUIClickTracking > MobileWikiAppAppearanceSettings > MobileWikiAppArticleSuggestions > MobileWikiAppCreateAccount > MobileWikiAppDailyStats > MobileWikiAppEdit > MobileWikiAppFeed > MobileWikiAppFeedConfigure > MobileWikiAppFindInPage > MobileWikiAppInstallReferrer > MobileWikiAppIntents > MobileWikiAppiOSFeed > MobileWikiAppiOSLoginAction > MobileWikiAppiOSReadingLists > MobileWikiAppiOSSearch > MobileWikiAppiOSSessions > MobileWikiAppiOSSettingAction > MobileWikiAppiOSUserHistory > MobileWikiAppLangSelect > MobileWikiAppLanguageSearching > MobileWikiAppLanguageSettings > MobileWikiAppLinkPreview > MobileWikiAppLogin > MobileWikiAppMediaGallery > MobileWikiAppNavMenu > MobileWikiAppNotificationInteraction > MobileWikiAppNotificationPreferences > MobileWikiAppOfflineLibrary > MobileWikiAppOnboarding > MobileWikiAppOnThisDay > MobileWikiAppPageScroll > MobileWikiAppProtectedEditAttempt > MobileWikiAppRandomizer > MobileWikiAppReadingLists > MobileWikiAppSavedPages > MobileWikiAppSearch > MobileWikiAppSessions > MobileWikiAppShareAFact > MobileWikiAppStuffHappens > MobileWikiAppTabs > MobileWikiAppToCInteraction > MobileWikiAppWidgets > MobileWikiAppWiktionaryPopup > MultimediaViewerAttribution > MultimediaViewerDimensions > MultimediaViewerDuration > MultimediaViewerNetworkPerformance > NavigationTiming > PageIssues > Popups > PrefUpdate > Print > QuickSurveyInitiation > QuickSurveysResponses > ReadingDepth > ResourceTiming > SaveTiming > SearchSatisfaction > SearchSatisfactionErrors >
Re: [Analytics] EventLogging Hive Refine currently stalled for some Schemas
Does "fixed" mean that the missing data already been backfilled? I'm seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the timespans between around 6am-16pm on November 13, and 7am-10am on November 12. On Thu, Nov 15, 2018 at 6:51 AM Andrew Otto wrote: > OH I’m sorry! There is a Phab task, and this is fixed. > https://phabricator.wikimedia.org/T209407 > > Very sorry, I should have updated and linked the task. > > On Thu, Nov 15, 2018 at 9:50 AM Gilles Dubuc wrote: > >> Is this issue still ongoing? Is there a corresponding Phabricator task to >> follow? >> >> On Tue, Nov 13, 2018 at 6:27 PM Andrew Otto wrote: >> >>> Hi all, >>> >>> Yesterday we upgraded the Hadoop cluster to a newer version. It seems >>> that along the way the job that imports EventLogging data into Hive has >>> started failing for some EventLogging datasets. I’m still investigating, >>> but it seems that tables (I believe ones that need to have their schema >>> modified during refinement) are failing refinement. The list of affected >>> tables at this moment is large, and will likely only grow as time goes on. >>> >>> We’re looking into this and will try to have things fixed ASAP. For now >>> you might expect Hive EventLogging data to be delayed. >>> >>> Sorry for the trouble, and we’ll keep you updated! >>> -Andrew Otto >>> Systems Engineer, WMF >>> >>> >>> >>> The list of currently affected tables is: >>> >>> AdvancedSearchRequest >>> CentralAuth >>> CentralNoticeBannerHistory >>> CentralNoticeImpression >>> CentralNoticeTiming >>> ChangesListFilterGrouping >>> ChangesListFilters >>> CitationUsage >>> CitationUsagePageLoad >>> ContentTranslation >>> ContentTranslationAbuseFilter >>> ContentTranslationCTA >>> ContentTranslationError >>> ContentTranslationSuggestion >>> CpuBenchmark >>> EchoInteraction >>> EchoMail >>> EditAttemptStep >>> EditConflict >>> EditorActivation >>> EUCCVisit >>> EventError >>> FlowReplies >>> GettingStartedRedirectImpression >>> GuidedTourButtonClick >>> GuidedTourExited >>> GuidedTourExternalLinkActivation >>> GuidedTourGuiderHidden >>> GuidedTourGuiderImpression >>> LandingPageImpression >>> MediaViewer >>> MediaWikiPingback >>> MobileAppCategorizationAttempts >>> MobileAppUploadAttempts >>> MobileWebMainMenuClickTracking >>> MobileWebSearch >>> MobileWebUIClickTracking >>> MobileWikiAppAppearanceSettings >>> MobileWikiAppArticleSuggestions >>> MobileWikiAppCreateAccount >>> MobileWikiAppDailyStats >>> MobileWikiAppEdit >>> MobileWikiAppFeed >>> MobileWikiAppFeedConfigure >>> MobileWikiAppFindInPage >>> MobileWikiAppInstallReferrer >>> MobileWikiAppIntents >>> MobileWikiAppiOSFeed >>> MobileWikiAppiOSLoginAction >>> MobileWikiAppiOSReadingLists >>> MobileWikiAppiOSSearch >>> MobileWikiAppiOSSessions >>> MobileWikiAppiOSSettingAction >>> MobileWikiAppiOSUserHistory >>> MobileWikiAppLangSelect >>> MobileWikiAppLanguageSearching >>> MobileWikiAppLanguageSettings >>> MobileWikiAppLinkPreview >>> MobileWikiAppLogin >>> MobileWikiAppMediaGallery >>> MobileWikiAppNavMenu >>> MobileWikiAppNotificationInteraction >>> MobileWikiAppNotificationPreferences >>> MobileWikiAppOfflineLibrary >>> MobileWikiAppOnboarding >>> MobileWikiAppOnThisDay >>> MobileWikiAppPageScroll >>> MobileWikiAppProtectedEditAttempt >>> MobileWikiAppRandomizer >>> MobileWikiAppReadingLists >>> MobileWikiAppSavedPages >>> MobileWikiAppSearch >>> MobileWikiAppSessions >>> MobileWikiAppShareAFact >>> MobileWikiAppStuffHappens >>> MobileWikiAppTabs >>> MobileWikiAppToCInteraction >>> MobileWikiAppWidgets >>> MobileWikiAppWiktionaryPopup >>> MultimediaViewerAttribution >>> MultimediaViewerDimensions >>> MultimediaViewerDuration >>> MultimediaViewerNetworkPerformance >>> NavigationTiming >>> PageIssues >>> Popups >>> PrefUpdate >>> Print >>> QuickSurveyInitiation >>> QuickSurveysResponses >>> ReadingDepth >>> ResourceTiming >>> SaveTiming >>> SearchSatisfaction >>> SearchSatisfactionErrors >>> ServerSideAccountCreation >>> TemplateWizard >>> TestSearchSatisfaction2 >>> TranslationRecommendationAPIRequests >>> TranslationRecommendationUIRequests >>> TranslationRecommendationUserAction >>> TwoColConflictConflict >>> UniversalLanguageSelector >>> UploadWizardErrorFlowEvent >>> UploadWizardExceptionFlowEvent >>> UploadWizardFlowEvent >>> UploadWizardStep >>> UploadWizardTutorialActions >>> UploadWizardUploadFlowEvent >>> VirtualPageView >>> WikidataCompletionSearchClicks >>> WikimediaBlogVisit >>> WikipediaPortal >>> WikipediaZeroUsage >>> WMDEBannerEvents >>> WMDEBannerSizeIssue >>> ___ >>> Analytics mailing list >>> Analytics@lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/analytics >>> >> ___ >> Analytics mailing list >> Analytics@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/analytics >> > -- Tilman Bayer Senior Analyst Wikimedia Foundation IRC (Freenode): HaeB
Re: [Analytics] EventLogging Hive Refine currently stalled for some Schemas
OH I’m sorry! There is a Phab task, and this is fixed. https://phabricator.wikimedia.org/T209407 Very sorry, I should have updated and linked the task. On Thu, Nov 15, 2018 at 9:50 AM Gilles Dubuc wrote: > Is this issue still ongoing? Is there a corresponding Phabricator task to > follow? > > On Tue, Nov 13, 2018 at 6:27 PM Andrew Otto wrote: > >> Hi all, >> >> Yesterday we upgraded the Hadoop cluster to a newer version. It seems >> that along the way the job that imports EventLogging data into Hive has >> started failing for some EventLogging datasets. I’m still investigating, >> but it seems that tables (I believe ones that need to have their schema >> modified during refinement) are failing refinement. The list of affected >> tables at this moment is large, and will likely only grow as time goes on. >> >> We’re looking into this and will try to have things fixed ASAP. For now >> you might expect Hive EventLogging data to be delayed. >> >> Sorry for the trouble, and we’ll keep you updated! >> -Andrew Otto >> Systems Engineer, WMF >> >> >> >> The list of currently affected tables is: >> >> AdvancedSearchRequest >> CentralAuth >> CentralNoticeBannerHistory >> CentralNoticeImpression >> CentralNoticeTiming >> ChangesListFilterGrouping >> ChangesListFilters >> CitationUsage >> CitationUsagePageLoad >> ContentTranslation >> ContentTranslationAbuseFilter >> ContentTranslationCTA >> ContentTranslationError >> ContentTranslationSuggestion >> CpuBenchmark >> EchoInteraction >> EchoMail >> EditAttemptStep >> EditConflict >> EditorActivation >> EUCCVisit >> EventError >> FlowReplies >> GettingStartedRedirectImpression >> GuidedTourButtonClick >> GuidedTourExited >> GuidedTourExternalLinkActivation >> GuidedTourGuiderHidden >> GuidedTourGuiderImpression >> LandingPageImpression >> MediaViewer >> MediaWikiPingback >> MobileAppCategorizationAttempts >> MobileAppUploadAttempts >> MobileWebMainMenuClickTracking >> MobileWebSearch >> MobileWebUIClickTracking >> MobileWikiAppAppearanceSettings >> MobileWikiAppArticleSuggestions >> MobileWikiAppCreateAccount >> MobileWikiAppDailyStats >> MobileWikiAppEdit >> MobileWikiAppFeed >> MobileWikiAppFeedConfigure >> MobileWikiAppFindInPage >> MobileWikiAppInstallReferrer >> MobileWikiAppIntents >> MobileWikiAppiOSFeed >> MobileWikiAppiOSLoginAction >> MobileWikiAppiOSReadingLists >> MobileWikiAppiOSSearch >> MobileWikiAppiOSSessions >> MobileWikiAppiOSSettingAction >> MobileWikiAppiOSUserHistory >> MobileWikiAppLangSelect >> MobileWikiAppLanguageSearching >> MobileWikiAppLanguageSettings >> MobileWikiAppLinkPreview >> MobileWikiAppLogin >> MobileWikiAppMediaGallery >> MobileWikiAppNavMenu >> MobileWikiAppNotificationInteraction >> MobileWikiAppNotificationPreferences >> MobileWikiAppOfflineLibrary >> MobileWikiAppOnboarding >> MobileWikiAppOnThisDay >> MobileWikiAppPageScroll >> MobileWikiAppProtectedEditAttempt >> MobileWikiAppRandomizer >> MobileWikiAppReadingLists >> MobileWikiAppSavedPages >> MobileWikiAppSearch >> MobileWikiAppSessions >> MobileWikiAppShareAFact >> MobileWikiAppStuffHappens >> MobileWikiAppTabs >> MobileWikiAppToCInteraction >> MobileWikiAppWidgets >> MobileWikiAppWiktionaryPopup >> MultimediaViewerAttribution >> MultimediaViewerDimensions >> MultimediaViewerDuration >> MultimediaViewerNetworkPerformance >> NavigationTiming >> PageIssues >> Popups >> PrefUpdate >> Print >> QuickSurveyInitiation >> QuickSurveysResponses >> ReadingDepth >> ResourceTiming >> SaveTiming >> SearchSatisfaction >> SearchSatisfactionErrors >> ServerSideAccountCreation >> TemplateWizard >> TestSearchSatisfaction2 >> TranslationRecommendationAPIRequests >> TranslationRecommendationUIRequests >> TranslationRecommendationUserAction >> TwoColConflictConflict >> UniversalLanguageSelector >> UploadWizardErrorFlowEvent >> UploadWizardExceptionFlowEvent >> UploadWizardFlowEvent >> UploadWizardStep >> UploadWizardTutorialActions >> UploadWizardUploadFlowEvent >> VirtualPageView >> WikidataCompletionSearchClicks >> WikimediaBlogVisit >> WikipediaPortal >> WikipediaZeroUsage >> WMDEBannerEvents >> WMDEBannerSizeIssue >> ___ >> Analytics mailing list >> Analytics@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/analytics >> > ___ > Analytics mailing list > Analytics@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/analytics > ___ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Re: [Analytics] EventLogging Hive Refine currently stalled for some Schemas
Is this issue still ongoing? Is there a corresponding Phabricator task to follow? On Tue, Nov 13, 2018 at 6:27 PM Andrew Otto wrote: > Hi all, > > Yesterday we upgraded the Hadoop cluster to a newer version. It seems > that along the way the job that imports EventLogging data into Hive has > started failing for some EventLogging datasets. I’m still investigating, > but it seems that tables (I believe ones that need to have their schema > modified during refinement) are failing refinement. The list of affected > tables at this moment is large, and will likely only grow as time goes on. > > We’re looking into this and will try to have things fixed ASAP. For now > you might expect Hive EventLogging data to be delayed. > > Sorry for the trouble, and we’ll keep you updated! > -Andrew Otto > Systems Engineer, WMF > > > > The list of currently affected tables is: > > AdvancedSearchRequest > CentralAuth > CentralNoticeBannerHistory > CentralNoticeImpression > CentralNoticeTiming > ChangesListFilterGrouping > ChangesListFilters > CitationUsage > CitationUsagePageLoad > ContentTranslation > ContentTranslationAbuseFilter > ContentTranslationCTA > ContentTranslationError > ContentTranslationSuggestion > CpuBenchmark > EchoInteraction > EchoMail > EditAttemptStep > EditConflict > EditorActivation > EUCCVisit > EventError > FlowReplies > GettingStartedRedirectImpression > GuidedTourButtonClick > GuidedTourExited > GuidedTourExternalLinkActivation > GuidedTourGuiderHidden > GuidedTourGuiderImpression > LandingPageImpression > MediaViewer > MediaWikiPingback > MobileAppCategorizationAttempts > MobileAppUploadAttempts > MobileWebMainMenuClickTracking > MobileWebSearch > MobileWebUIClickTracking > MobileWikiAppAppearanceSettings > MobileWikiAppArticleSuggestions > MobileWikiAppCreateAccount > MobileWikiAppDailyStats > MobileWikiAppEdit > MobileWikiAppFeed > MobileWikiAppFeedConfigure > MobileWikiAppFindInPage > MobileWikiAppInstallReferrer > MobileWikiAppIntents > MobileWikiAppiOSFeed > MobileWikiAppiOSLoginAction > MobileWikiAppiOSReadingLists > MobileWikiAppiOSSearch > MobileWikiAppiOSSessions > MobileWikiAppiOSSettingAction > MobileWikiAppiOSUserHistory > MobileWikiAppLangSelect > MobileWikiAppLanguageSearching > MobileWikiAppLanguageSettings > MobileWikiAppLinkPreview > MobileWikiAppLogin > MobileWikiAppMediaGallery > MobileWikiAppNavMenu > MobileWikiAppNotificationInteraction > MobileWikiAppNotificationPreferences > MobileWikiAppOfflineLibrary > MobileWikiAppOnboarding > MobileWikiAppOnThisDay > MobileWikiAppPageScroll > MobileWikiAppProtectedEditAttempt > MobileWikiAppRandomizer > MobileWikiAppReadingLists > MobileWikiAppSavedPages > MobileWikiAppSearch > MobileWikiAppSessions > MobileWikiAppShareAFact > MobileWikiAppStuffHappens > MobileWikiAppTabs > MobileWikiAppToCInteraction > MobileWikiAppWidgets > MobileWikiAppWiktionaryPopup > MultimediaViewerAttribution > MultimediaViewerDimensions > MultimediaViewerDuration > MultimediaViewerNetworkPerformance > NavigationTiming > PageIssues > Popups > PrefUpdate > Print > QuickSurveyInitiation > QuickSurveysResponses > ReadingDepth > ResourceTiming > SaveTiming > SearchSatisfaction > SearchSatisfactionErrors > ServerSideAccountCreation > TemplateWizard > TestSearchSatisfaction2 > TranslationRecommendationAPIRequests > TranslationRecommendationUIRequests > TranslationRecommendationUserAction > TwoColConflictConflict > UniversalLanguageSelector > UploadWizardErrorFlowEvent > UploadWizardExceptionFlowEvent > UploadWizardFlowEvent > UploadWizardStep > UploadWizardTutorialActions > UploadWizardUploadFlowEvent > VirtualPageView > WikidataCompletionSearchClicks > WikimediaBlogVisit > WikipediaPortal > WikipediaZeroUsage > WMDEBannerEvents > WMDEBannerSizeIssue > ___ > Analytics mailing list > Analytics@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/analytics > ___ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
