Hello, Not all data sources are populated at the same time, the data on Druid is ingested twice, once per hour and once daily looking 4 days back. Data should appear once daily job runs for the "holes" missing.
Thanks, Nuria On Thu, Nov 15, 2018 at 7:49 AM Andrew Otto <o...@wikimedia.org> wrote: > > Does "fixed" mean that the missing data already been backfilled? I’m > seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the > timespans between around 6am-16pm on November 13, and 7am-10am on November > 12. > > Hm. Fixed means the data has been refined to Hive. I didn’t check on > Druid EL data. Marcel, can you check up on that? This sounds related, we > might need to figure out how to re run the Druid ingestion if/when base > data partitions change. > > > > On Thu, Nov 15, 2018 at 10:17 AM Tilman Bayer <tba...@wikimedia.org> > wrote: > >> Hi Andrew, >> >> PS, offtopic: reminder that data-analy...@wikimedia.org has been >> deprecated/retired a while ago; our team can be reached at >> product-analyt...@wikimedia.org (CCed) >> >> On Thu, Nov 15, 2018 at 7:13 AM Tilman Bayer <tba...@wikimedia.org> >> wrote: >> >>> Does "fixed" mean that the missing data already been backfilled? I'm >>> seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the >>> timespans between around 6am-16pm on November 13, and 7am-10am on November >>> 12. >>> >>> On Thu, Nov 15, 2018 at 6:51 AM Andrew Otto <o...@wikimedia.org> wrote: >>> >>>> OH I’m sorry! There is a Phab task, and this is fixed. >>>> https://phabricator.wikimedia.org/T209407 >>>> >>>> Very sorry, I should have updated and linked the task. >>>> >>>> On Thu, Nov 15, 2018 at 9:50 AM Gilles Dubuc <gil...@wikimedia.org> >>>> wrote: >>>> >>>>> Is this issue still ongoing? Is there a corresponding Phabricator task >>>>> to follow? >>>>> >>>>> On Tue, Nov 13, 2018 at 6:27 PM Andrew Otto <o...@wikimedia.org> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> Yesterday we upgraded the Hadoop cluster to a newer version. It >>>>>> seems that along the way the job that imports EventLogging data into Hive >>>>>> has started failing for some EventLogging datasets. I’m still >>>>>> investigating, but it seems that tables (I believe ones that need to have >>>>>> their schema modified during refinement) are failing refinement. The >>>>>> list >>>>>> of affected tables at this moment is large, and will likely only grow as >>>>>> time goes on. >>>>>> >>>>>> We’re looking into this and will try to have things fixed ASAP. For >>>>>> now you might expect Hive EventLogging data to be delayed. >>>>>> >>>>>> Sorry for the trouble, and we’ll keep you updated! >>>>>> -Andrew Otto >>>>>> Systems Engineer, WMF >>>>>> >>>>>> >>>>>> >>>>>> The list of currently affected tables is: >>>>>> >>>>>> AdvancedSearchRequest >>>>>> CentralAuth >>>>>> CentralNoticeBannerHistory >>>>>> CentralNoticeImpression >>>>>> CentralNoticeTiming >>>>>> ChangesListFilterGrouping >>>>>> ChangesListFilters >>>>>> CitationUsage >>>>>> CitationUsagePageLoad >>>>>> ContentTranslation >>>>>> ContentTranslationAbuseFilter >>>>>> ContentTranslationCTA >>>>>> ContentTranslationError >>>>>> ContentTranslationSuggestion >>>>>> CpuBenchmark >>>>>> EchoInteraction >>>>>> EchoMail >>>>>> EditAttemptStep >>>>>> EditConflict >>>>>> EditorActivation >>>>>> EUCCVisit >>>>>> EventError >>>>>> FlowReplies >>>>>> GettingStartedRedirectImpression >>>>>> GuidedTourButtonClick >>>>>> GuidedTourExited >>>>>> GuidedTourExternalLinkActivation >>>>>> GuidedTourGuiderHidden >>>>>> GuidedTourGuiderImpression >>>>>> LandingPageImpression >>>>>> MediaViewer >>>>>> MediaWikiPingback >>>>>> MobileAppCategorizationAttempts >>>>>> MobileAppUploadAttempts >>>>>> MobileWebMainMenuClickTracking >>>>>> MobileWebSearch >>>>>> MobileWebUIClickTracking >>>>>> MobileWikiAppAppearanceSettings >>>>>> MobileWikiAppArticleSuggestions >>>>>> MobileWikiAppCreateAccount >>>>>> MobileWikiAppDailyStats >>>>>> MobileWikiAppEdit >>>>>> MobileWikiAppFeed >>>>>> MobileWikiAppFeedConfigure >>>>>> MobileWikiAppFindInPage >>>>>> MobileWikiAppInstallReferrer >>>>>> MobileWikiAppIntents >>>>>> MobileWikiAppiOSFeed >>>>>> MobileWikiAppiOSLoginAction >>>>>> MobileWikiAppiOSReadingLists >>>>>> MobileWikiAppiOSSearch >>>>>> MobileWikiAppiOSSessions >>>>>> MobileWikiAppiOSSettingAction >>>>>> MobileWikiAppiOSUserHistory >>>>>> MobileWikiAppLangSelect >>>>>> MobileWikiAppLanguageSearching >>>>>> MobileWikiAppLanguageSettings >>>>>> MobileWikiAppLinkPreview >>>>>> MobileWikiAppLogin >>>>>> MobileWikiAppMediaGallery >>>>>> MobileWikiAppNavMenu >>>>>> MobileWikiAppNotificationInteraction >>>>>> MobileWikiAppNotificationPreferences >>>>>> MobileWikiAppOfflineLibrary >>>>>> MobileWikiAppOnboarding >>>>>> MobileWikiAppOnThisDay >>>>>> MobileWikiAppPageScroll >>>>>> MobileWikiAppProtectedEditAttempt >>>>>> MobileWikiAppRandomizer >>>>>> MobileWikiAppReadingLists >>>>>> MobileWikiAppSavedPages >>>>>> MobileWikiAppSearch >>>>>> MobileWikiAppSessions >>>>>> MobileWikiAppShareAFact >>>>>> MobileWikiAppStuffHappens >>>>>> MobileWikiAppTabs >>>>>> MobileWikiAppToCInteraction >>>>>> MobileWikiAppWidgets >>>>>> MobileWikiAppWiktionaryPopup >>>>>> MultimediaViewerAttribution >>>>>> MultimediaViewerDimensions >>>>>> MultimediaViewerDuration >>>>>> MultimediaViewerNetworkPerformance >>>>>> NavigationTiming >>>>>> PageIssues >>>>>> Popups >>>>>> PrefUpdate >>>>>> Print >>>>>> QuickSurveyInitiation >>>>>> QuickSurveysResponses >>>>>> ReadingDepth >>>>>> ResourceTiming >>>>>> SaveTiming >>>>>> SearchSatisfaction >>>>>> SearchSatisfactionErrors >>>>>> ServerSideAccountCreation >>>>>> TemplateWizard >>>>>> TestSearchSatisfaction2 >>>>>> TranslationRecommendationAPIRequests >>>>>> TranslationRecommendationUIRequests >>>>>> TranslationRecommendationUserAction >>>>>> TwoColConflictConflict >>>>>> UniversalLanguageSelector >>>>>> UploadWizardErrorFlowEvent >>>>>> UploadWizardExceptionFlowEvent >>>>>> UploadWizardFlowEvent >>>>>> UploadWizardStep >>>>>> UploadWizardTutorialActions >>>>>> UploadWizardUploadFlowEvent >>>>>> VirtualPageView >>>>>> WikidataCompletionSearchClicks >>>>>> WikimediaBlogVisit >>>>>> WikipediaPortal >>>>>> WikipediaZeroUsage >>>>>> WMDEBannerEvents >>>>>> WMDEBannerSizeIssue >>>>>> _______________________________________________ >>>>>> Analytics mailing list >>>>>> Analytics@lists.wikimedia.org >>>>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>>>>> >>>>> _______________________________________________ >>>>> Analytics mailing list >>>>> Analytics@lists.wikimedia.org >>>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>>>> >>>> >>> >>> -- >>> Tilman Bayer >>> Senior Analyst >>> Wikimedia Foundation >>> IRC (Freenode): HaeB >>> >> >> >> -- >> Tilman Bayer >> Senior Analyst >> Wikimedia Foundation >> IRC (Freenode): HaeB >> > _______________________________________________ > Analytics mailing list > Analytics@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/analytics >
_______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
