[jira] [Deleted] (MESOS-9062) Mohamedvolt

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-9062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu deleted MESOS-9062:
-


> Mohamedvolt
> ---
>
> Key: MESOS-9062
> URL: https://issues.apache.org/jira/browse/MESOS-9062
> Project: Mesos
>  Issue Type: Story
>Reporter: Mohamedvolt 
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Deleted] (MESOS-9063) Mohamedvolt

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu deleted MESOS-9063:
-


> Mohamedvolt
> ---
>
> Key: MESOS-9063
> URL: https://issues.apache.org/jira/browse/MESOS-9063
> Project: Mesos
>  Issue Type: Bug
>Reporter: Mohamedvolt 
>Priority: Minor
>
> rformat s.sperdher*strong text*w



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Deleted] (MESOS-9064) https://confluence.atlassian.com/formated/all/datajirasoftwareserver076/saving-your-search-as-a-filter-941595927.html#managing_filters

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-9064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu deleted MESOS-9064:
-


> https://confluence.atlassian.com/formated/all/datajirasoftwareserver076/saving-your-search-as-a-filter-941595927.html#managing_filters
> --
>
> Key: MESOS-9064
> URL: https://issues.apache.org/jira/browse/MESOS-9064
> Project: Mesos
>  Issue Type: Bug
>Reporter: Mohamedvolt 
>Priority: Major
>  Labels: 12311242, all, data, rformwted.bat
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Deleted] (MESOS-9060) relesalldataLinked ApplicationsASF JIRADashboardsProjectsIssuesBoards CreateSearchGive feedback to AtlassianHelpUser profile for Mohamedvolt Browse ProjectsPROJECT TYPESA

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-9060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu deleted MESOS-9060:
-


> relesalldataLinked ApplicationsASF JIRADashboardsProjectsIssuesBoards 
> CreateSearchGive feedback to AtlassianHelpUser profile for Mohamedvolt Browse 
> ProjectsPROJECT TYPESAll Project TypesSoftwareBusinessCATEGORIESAll 
> CategoriesMetaModel
> ---
>
> Key: MESOS-9060
> URL: https://issues.apache.org/jira/browse/MESOS-9060
> Project: Mesos
>  Issue Type: Task
>Reporter: Mohamedvolt 
>Priority: Major
>
> r.bather:Linked Applications
> ASF JIRA
> Dashboards
> Projects
> Issues
> Boards
>  
> Create
> Search
> Give feedback to Atlassian
> Help
> User profile for Mohamedvolt 
> Browse Projects
> PROJECT TYPES
> All Project Types
> Software
> Business
> CATEGORIES
> All Categories
> MetaModel
> Quickstep
> Abdera
> Accumulo
> Ace
> ActiveMQ
> Airavata
> Airflow
> Ambari
> Ant
> Any23
> Apache Software Foundation
> Apex
> Archiva
> Aries
> Arrow
> AsterixDB
> Aurora
> Avro
> Axis
> Bahir
> Batik
> Beam
> BigTop
> BookKeeper
> Brooklyn
> Buildr
> BVal
> C++ Standard Library
> Calcite
> Camel
> CarbonData
> Cassandra
> Cayenne
> Chemistry
> Chukwa
> Click
> Cloudstack
> Cocoon
> Commons
> Continuum
> Cordova
> CouchDB
> Creadur
> Crunch
> CXF
> DB
> DeltaCloud
> Deltaspike
> Devicemap
> Directory
> DistributedLog
> Eagle
> ESME
> Etch
> Felix
> Fineract
> Flex
> Flink
> Flume
> Formatting Objects Processor
> Forrest
> Freemarker
> Geode
> Geronimo
> Giraph
> Gora
> Gossip
> Groovy
> Guacamole
> Gump
> Hadoop
> Hama
> Helix
> Hivemall
> HTTP Server
> HttpComponents
> Ignite
> Impala
> Incubator
> Infrastructure
> Isis
> Jackrabbit
> James
> jclouds
> Jena
> JSPWiki
> jUDDI
> Kafka
> Karaf
> Knox
> Kudu
> Kylin
> Labs
> Lens
> Libcloud
> Logging
> Lucene
> Lucy
> MADlib
> Mahout
> ManifoldCF
> Maven
> Mesos
> Metron
> MINA
> Mnemonic
> MRUnit
> MyFaces
> Mynewt
> Myriad
> NiFi
> Nutch
> ODE
> OFBiz
> Olingo
> Onami
> OODT
> Oozie
> Open Climate
> OpenEJB
> OpenJPA
> Openmeetings
> OpenNLP
> OpenWebBeans
> Orc
> PDFBox
> Phoenix
> Pivot
> Polygene
> Portals
> Qpid
> Ranger
> REEF
> Retired
> River
> Roller
> Samza
> Sentry
> ServiceMix
> Shindig
> Shiro
> SIS
> Sling
> Sqoop
> Steve
> Storm
> Stratos
> Struts Framework
> Synapse
> Syncope
> SystemML
> Tapestry
> Taverna
> Tez
> Thrift
> Tika
> Tiles Framework
> Tomcat
> Traffic Server
> Trafodion
> Turbine
> Tuscany
> Twill
> UIMA
> Usergrid
> Velocity
> Web Services
> Whimsy
> Wicket
> Xalan
> XML
> XMLBeans
> Yetus
> Zeppelin
> ZooKeeper
> No Category
> Recent Projects
> All Project Types - Retired
> Contains text...
> Project   Key Project TypeProject LeadProject Category
> URL
>   Agila   AGILA   SoftwareGeir Magnusson Jr   Retired No URL
>   Addressing  ADDRSoftwareGlen DanielsRetired 
> http://axis.apache.org/
>   ALOIS   ALOIS   SoftwareScott Deboy Retired 
> http://incubator.apache.org/alois
>   AltRMI  ARMISoftwarePaul HammantRetired 
> http://incubator.apache.org/projects/altrmi
>   Apache Blur BLURSoftwarePatrick HuntRetired No URL
>   Apache Climate Model Diagnostic AnalyzerCMDASoftware
> Chris A. Mattmann   Retired https://incubator.apache.org/
>   Apache ConcertedCONCERTED   SoftwareAtri Sharma 
> Retired http://incubator.apache.org
>   Apache Oltu OLTUSoftwareAntonio Sanso   Retired 
> http://oltu.apache.org/
>   Apache Ripple   RIPPLE  SoftwareThe Ripple CommunityRetired 
> http://ripple.incubator.apache.org
>   Apache Whirr (retired)  WHIRR   SoftwareAndrew BayerRetired 
> https://attic.apache.org/projects/whirr.html
>   Avalon  AVALON  SoftwareCarsten ZiegelerRetired 
> http://excalibur.apache.org/
>   Avalon Castle   AVNSHARPSoftwarehamilton verissimo  
> Retired http://avalon.apache.org/
>   Avalon Merlin Runtime   RUNTIME SoftwareStephen McConnell   
> Retired http://avalon.apache.org/
>   Avalon Merlin StudioSTUDIO  SoftwareAndreas Oberhack
> Retired http://avalon.apache.org/
>   Avalon Metro CentralCENTRAL SoftwareNiclas Hedhman  Retired 
> http://avalon.apache.org/
>   Avalon Metro Planet PLANET  SoftwareStephen McConnell   
> Retired http://avalon.apache.org/
>   Avalon Metro Tools  TOOLS   SoftwareNiclas Hedhman  Retired 
> http://avalon.apache.org/
>   Avalon Phoenix  PNIX

[jira] [Deleted] (MESOS-9061) rform8atedher/MESOS-3937

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-9061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu deleted MESOS-9061:
-


> rform8atedher/MESOS-3937
> 
>
> Key: MESOS-9061
> URL: https://issues.apache.org/jira/browse/MESOS-9061
> Project: Mesos
>  Issue Type: Story
>Reporter: Mohamedvolt 
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (MESOS-9038) Archiver utility extracts links within subdirectories incorrectly

[Joseph Wu (JIRA)]

Joseph Wu created MESOS-9038:


 Summary: Archiver utility extracts links within subdirectories 
incorrectly
 Key: MESOS-9038
 URL: https://issues.apache.org/jira/browse/MESOS-9038
 Project: Mesos
  Issue Type: Bug
Reporter: Joseph Wu
Assignee: Joseph Wu


The fix for MESOS-9008 did not correctly fix extraction of links with the 
archiver utility.

Two problems:
1) Links that were originally relative within the archiver are transformed to 
point to absolute locations after extraction.  The link targets should remain 
relative.
2) Links within subdirectories (i.e. path/to/link -> path/to/target) will lose 
part of the target path upon extraction (i.e. path/to/link -> target)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (MESOS-9008) Fetcher fails to extract some archives containing hardlinks

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu reassigned MESOS-9008:


Assignee: Joseph Wu

> Fetcher fails to extract some archives containing hardlinks
> ---
>
> Key: MESOS-9008
> URL: https://issues.apache.org/jira/browse/MESOS-9008
> Project: Mesos
>  Issue Type: Bug
>  Components: fetcher
>Affects Versions: 1.7.0
>Reporter: Benjamin Bannier
>Assignee: Joseph Wu
>Priority: Blocker
>
> We recently switched the infrastructure to e.g., extract archives to 
> libarchive which seems to have narrower support for e.g., hardlinks in 
> archives, see e.g., [https://github.com/libarchive/libarchive/wiki/Hardlinks] 
> upstream (likely outdated).
> In a particular case, we tried to extract 
> https://artifacts.elastic.co/downloads/kibana/kibana-5.6.5-linux-x86_64.tar.gz
>  which the fetcher successfully extracted in e.g., 1.6.0, but which now leads 
> to failures like
> {noformat}
> W0619 08:53:38.00  4610 fetcher.cpp:913] Begin fetcher log (stderr in 
> sandbox) for container d39bc16c-e6c6-440f-a82c-ad26332a1b36 from running 
> command: 
> /opt/mesosphere/packages/mesos--95f27ab971fb1d03bc1277f6a16e63a9815e1a61/libexec/mesos/mesos-fetcher
> I0619 08:53:34.620458  7571 fetcher.cpp:560] Fetcher Info: 
> {"cache_directory":"\/tmp\/mesos\/fetch\/nobody","items":[{"action":"DOWNLOAD_AND_CACHE","cache_filename":"c29-kibana-5.6__64.tar.gz","uri":{"cache":true,"executable":false,"extract":true,"value":"https:\/\/artifacts.elastic.co\/downloads\/kibana\/kibana-5.6.5-linux-x86_64.tar.gz"}},{"action":"DOWNLOAD_AND_CACHE","cache_filename":"c30-x-pack-5.6.5.zip","uri":{"cache":true,"executable":false,"extract":false,"value":"https:\/\/artifacts.elastic.co\/downloads\/packs\/x-pack\/x-pack-5.6.5.zip"}}],"sandbox_directory":"\/var\/lib\/mesos\/slave\/slaves\/7f2e186c-c748-493d-82e4-644b5b23c9bb-S6\/frameworks\/7f2e186c-c748-493d-82e4-644b5b23c9bb-0001\/executors\/kibana.3f6c7799-739e-11e8-a678-168288e5aa33\/runs\/d39bc16c-e6c6-440f-a82c-ad26332a1b36","stall_timeout":{"nanoseconds":600},"user":"nobody"}
> I0619 08:53:34.623996  7571 fetcher.cpp:457] Fetching URI 
> 'https://artifacts.elastic.co/downloads/kibana/kibana-5.6.5-linux-x86_64.tar.gz'
> I0619 08:53:34.624017  7571 fetcher.cpp:431] Downloading into cache
> I0619 08:53:34.624027  7571 fetcher.cpp:225] Fetching URI 
> 'https://artifacts.elastic.co/downloads/kibana/kibana-5.6.5-linux-x86_64.tar.gz'
> I0619 08:53:34.624043  7571 fetcher.cpp:175] Downloading resource from 
> 'https://artifacts.elastic.co/downloads/kibana/kibana-5.6.5-linux-x86_64.tar.gz'
>  to '/tmp/mesos/fetch/nobody/c29-kibana-5.6__64.tar.gz'
> I0619 08:53:38.634416  7571 fetcher.cpp:350] Fetching from cache
> E0619 08:53:38.686941  7571 fetcher.cpp:613] EXIT with status 1: Failed to 
> fetch 
> 'https://artifacts.elastic.co/downloads/kibana/kibana-5.6.5-linux-x86_64.tar.gz':
>  Failed to extract archive 
> '/tmp/mesos/fetch/nobody/c29-kibana-5.6__64.tar.gz' to 
> '/var/lib/mesos/slave/slaves/7f2e186c-c748-493d-82e4-644b5b23c9bb-S6/frameworks/7f2e186c-c748-493d-82e4-644b5b23c9bb-0001/executors/kibana.3f6c7799-739e-11e8-a678-168288e5aa33/runs/d39bc16c-e6c6-440f-a82c-ad26332a1b36':
>  Failed to write archive header: Hard-link target 
> 'kibana-5.6.5-linux-x86_64/node_modules/svgo/node_modules/js-yaml/bin/js-yaml.js'
>  does not exist.
> End fetcher log for container d39bc16c-e6c6-440f-a82c-ad26332a1b36
> E0619 08:53:38.00  4610 fetcher.cpp:571] Failed to run mesos-fetcher: 
> Failed to fetch all URIs for container 
> 'd39bc16c-e6c6-440f-a82c-ad26332a1b36': exited with status 1
> E0619 08:53:38.00  4608 slave.cpp:6180] Container 
> 'd39bc16c-e6c6-440f-a82c-ad26332a1b36' for executor 
> 'kibana.3f6c7799-739e-11e8-a678-168288e5aa33' of framework 
> 7f2e186c-c748-493d-82e4-644b5b23c9bb-0001 failed to start: Failed to fetch 
> all URIs for container 'd39bc16c-e6c6-440f-a82c-ad26332a1b36': exited with 
> status 1
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-9008) Fetcher fails to extract some archives containing hardlinks

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16518613#comment-16518613
 ] 

Joseph Wu commented on MESOS-9008:
--

I've tried renaming the tarball and extracting from a different directory.  
Both of which appear to work with the archiver code.  This was done on Ubuntu 
16 (Autotools) and OSX (CMake)...

Oddly, the tarball took about 10 minutes to download (I'd expect a few seconds 
with my connection), but that's probably irrelevant.

> Fetcher fails to extract some archives containing hardlinks
> ---
>
> Key: MESOS-9008
> URL: https://issues.apache.org/jira/browse/MESOS-9008
> Project: Mesos
>  Issue Type: Bug
>  Components: fetcher
>Affects Versions: 1.7.0
>Reporter: Benjamin Bannier
>Priority: Blocker
>
> We recently switched the infrastructure to e.g., extract archives to 
> libarchive which seems to have narrower support for e.g., hardlinks in 
> archives, see e.g., [https://github.com/libarchive/libarchive/wiki/Hardlinks] 
> upstream (likely outdated).
> In a particular case, we tried to extract 
> https://artifacts.elastic.co/downloads/kibana/kibana-5.6.5-linux-x86_64.tar.gz
>  which the fetcher successfully extracted in e.g., 1.6.0, but which now leads 
> to failures like
> {noformat}
> W0619 08:53:38.00  4610 fetcher.cpp:913] Begin fetcher log (stderr in 
> sandbox) for container d39bc16c-e6c6-440f-a82c-ad26332a1b36 from running 
> command: 
> /opt/mesosphere/packages/mesos--95f27ab971fb1d03bc1277f6a16e63a9815e1a61/libexec/mesos/mesos-fetcher
> I0619 08:53:34.620458  7571 fetcher.cpp:560] Fetcher Info: 
> {"cache_directory":"\/tmp\/mesos\/fetch\/nobody","items":[{"action":"DOWNLOAD_AND_CACHE","cache_filename":"c29-kibana-5.6__64.tar.gz","uri":{"cache":true,"executable":false,"extract":true,"value":"https:\/\/artifacts.elastic.co\/downloads\/kibana\/kibana-5.6.5-linux-x86_64.tar.gz"}},{"action":"DOWNLOAD_AND_CACHE","cache_filename":"c30-x-pack-5.6.5.zip","uri":{"cache":true,"executable":false,"extract":false,"value":"https:\/\/artifacts.elastic.co\/downloads\/packs\/x-pack\/x-pack-5.6.5.zip"}}],"sandbox_directory":"\/var\/lib\/mesos\/slave\/slaves\/7f2e186c-c748-493d-82e4-644b5b23c9bb-S6\/frameworks\/7f2e186c-c748-493d-82e4-644b5b23c9bb-0001\/executors\/kibana.3f6c7799-739e-11e8-a678-168288e5aa33\/runs\/d39bc16c-e6c6-440f-a82c-ad26332a1b36","stall_timeout":{"nanoseconds":600},"user":"nobody"}
> I0619 08:53:34.623996  7571 fetcher.cpp:457] Fetching URI 
> 'https://artifacts.elastic.co/downloads/kibana/kibana-5.6.5-linux-x86_64.tar.gz'
> I0619 08:53:34.624017  7571 fetcher.cpp:431] Downloading into cache
> I0619 08:53:34.624027  7571 fetcher.cpp:225] Fetching URI 
> 'https://artifacts.elastic.co/downloads/kibana/kibana-5.6.5-linux-x86_64.tar.gz'
> I0619 08:53:34.624043  7571 fetcher.cpp:175] Downloading resource from 
> 'https://artifacts.elastic.co/downloads/kibana/kibana-5.6.5-linux-x86_64.tar.gz'
>  to '/tmp/mesos/fetch/nobody/c29-kibana-5.6__64.tar.gz'
> I0619 08:53:38.634416  7571 fetcher.cpp:350] Fetching from cache
> E0619 08:53:38.686941  7571 fetcher.cpp:613] EXIT with status 1: Failed to 
> fetch 
> 'https://artifacts.elastic.co/downloads/kibana/kibana-5.6.5-linux-x86_64.tar.gz':
>  Failed to extract archive 
> '/tmp/mesos/fetch/nobody/c29-kibana-5.6__64.tar.gz' to 
> '/var/lib/mesos/slave/slaves/7f2e186c-c748-493d-82e4-644b5b23c9bb-S6/frameworks/7f2e186c-c748-493d-82e4-644b5b23c9bb-0001/executors/kibana.3f6c7799-739e-11e8-a678-168288e5aa33/runs/d39bc16c-e6c6-440f-a82c-ad26332a1b36':
>  Failed to write archive header: Hard-link target 
> 'kibana-5.6.5-linux-x86_64/node_modules/svgo/node_modules/js-yaml/bin/js-yaml.js'
>  does not exist.
> End fetcher log for container d39bc16c-e6c6-440f-a82c-ad26332a1b36
> E0619 08:53:38.00  4610 fetcher.cpp:571] Failed to run mesos-fetcher: 
> Failed to fetch all URIs for container 
> 'd39bc16c-e6c6-440f-a82c-ad26332a1b36': exited with status 1
> E0619 08:53:38.00  4608 slave.cpp:6180] Container 
> 'd39bc16c-e6c6-440f-a82c-ad26332a1b36' for executor 
> 'kibana.3f6c7799-739e-11e8-a678-168288e5aa33' of framework 
> 7f2e186c-c748-493d-82e4-644b5b23c9bb-0001 failed to start: Failed to fetch 
> all URIs for container 'd39bc16c-e6c6-440f-a82c-ad26332a1b36': exited with 
> status 1
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-8614) DefaultExecutorTests occassionally crash in the V1 Scheduler code

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448746#comment-16448746
 ] 

Joseph Wu commented on MESOS-8614:
--

Modified the JIRA title as this also appears in {{KillMultipleTasks/0}} (and 
probably can happen in any of the same tests using the V1 Scheduler mock).

> DefaultExecutorTests occassionally crash in the V1 Scheduler code
> -
>
> Key: MESOS-8614
> URL: https://issues.apache.org/jira/browse/MESOS-8614
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 1.5.0
>Reporter: Chun-Hung Hsiao
>Priority: Major
>  Labels: flaky-test, mesosphere
> Attachments: KillMultipleTasks-badrun.txt, consoleText.1.log, 
> consoleText.2.log, consoleText.3.log
>
>
> Occasionally the {{DefaultExecutorTest.ResourceLimitation/0}} and 
> {{DefaultExecutorTest.ROOT_ContainerStatusForTask/0}} would crash with the 
> following logs:
> {noformat}
> I*** Aborted at 1519639358 (unix time) try "date -d @1519639358" if you are 
> using GNU date ***
> 0226 10:02:38.030114 21366 task_status_update_manager.cpp:538] Cleaning up 
> status update stream for task a332e0b5-a713-47b2-85d8-358ce6a4118a of 
> framework 507afc07-d395-4e76-aa11-4562ae07a9b3-
> I0226 10:02:38.029911 21370 gc.cpp:90] Scheduling 
> '/tmp/ROOT_DOCKER_DockerAndMesosContainerizers_DefaultExecutorTest_ResourceLimitation_0_UVxsKT/slaves/507afc07-d395-4e76-aa11-4562ae07a9b3-S0/frameworks/507afc07-d395-4e76-aa11-4562ae07a9b3-/executors/default/runs/37678c9e-fc27-40fa-8d26-b540ff88a381'
>  for gc 6.9968157333days in the future
> I0226 10:02:38.030480 21370 gc.cpp:90] Scheduling 
> '/tmp/ROOT_DOCKER_DockerAndMesosContainerizers_DefaultExecutorTest_ResourceLimitation_0_UVxsKT/slaves/507afc07-d395-4e76-aa11-4562ae07a9b3-S0/frameworks/507afc07-d395-4e76-aa11-4562ae07a9b3-/executors/default'
>  for gc 6.9968157333days in the future
> I0226 10:02:38.030591 21370 gc.cpp:90] Scheduling 
> '/tmp/ROOT_DOCKER_DockerAndMesosContainerizers_DefaultExecutorTest_ResourceLimitation_0_UVxsKT/slaves/507afc07-d395-4e76-aa11-4562ae07a9b3-S0/frameworks/507afc07-d395-4e76-aa11-4562ae07a9b3-'
>  for gc 6.9968157333days in the future
> PC: @ 0x7f9b6df74eb3 mesos::v1::scheduler::Mesos::send()
> *** SIGSEGV (@0x0) received by PID 32110 (TID 0x7f9b626a9700) from PID 0; 
> stack trace: ***
> @ 0x7f9b3717b9c2 (unknown)
> @ 0x7f9b37180689 (unknown)
> @ 0x7f9b371743e8 (unknown)
> @ 0x7f9b6b7d3670 (unknown)
> @ 0x7f9b6df74eb3 mesos::v1::scheduler::Mesos::send()
> @ 0x55a24270c0f6 
> _ZNK5mesos8internal5tests2v19scheduler23SendAcknowledgeActionP2INS_2v111FrameworkIDENS5_7AgentIDEE10gmock_ImplIFvPNS5_9scheduler5MesosERKNSA_12Event_UpdateEEE17gmock_PerformImplISC_SF_N7testing8internal12ExcessiveArgESL_SL_SL_SL_SL_SL_SL_EEvRKSt5tupleIJSC_SF_EET_T0_T1_T2_T3_T4_T5_T6_T7_T8_
> @ 0x55a24270c26a 
> _ZN5mesos8internal5tests2v19scheduler23SendAcknowledgeActionP2INS_2v111FrameworkIDENS5_7AgentIDEE10gmock_ImplIFvPNS5_9scheduler5MesosERKNSA_12Event_UpdateEEE7PerformERKSt5tupleIJSC_SF_EE
> @ 0x55a2425fcc1e 
> _ZN7testing8internal12DoBothActionI17PromiseArgActionPILi1EPN7process7PromiseIN5mesos2v19scheduler12Event_UpdateNS5_8internal5tests2v19scheduler23SendAcknowledgeActionP2INS6_11FrameworkIDENS6_7AgentID4ImplIFvPNS7_5MesosERKS8_EE7PerformERKSt5tupleIJSN_SP_EE
> @ 0x55a24262e2b7 
> testing::internal::FunctionMockerBase<>::UntypedPerformAction()
> @ 0x55a2438a2d19 
> testing::internal::UntypedFunctionMockerBase::UntypedInvokeWith()
> @ 0x55a24270f27a 
> mesos::internal::tests::scheduler::MockHTTPScheduler<>::events()
> @ 0x55a24268aae3 std::_Function_handler<>::_M_invoke()
> @ 0x7f9b6df78bf8 process::AsyncExecutorProcess::execute<>()
> @ 0x7f9b6df8155d 
> _ZNO6lambda12CallableOnceIFvPN7process11ProcessBaseEEE10CallableFnINS_8internal7PartialIZNS1_8dispatchI7NothingNS1_20AsyncExecutorProcessERKSt8functionIFvRKSt5queueIN5mesos2v19scheduler5EventESt5dequeISH_SaISH_ESL_SR_RSL_EENS1_6FutureIT_EERKNS1_3PIDIT0_EEMSX_FSU_T1_T2_EOT3_OT4_EUlSt10unique_ptrINS1_7PromiseISA_EESt14default_deleteIS1B_EEOSP_OSL_S3_E_JS1E_SP_SL_St12_PlaceholderILi1EEclEOS3_
> @ 0x7f9b6eb3c1f1 process::ProcessBase::consume()
> @ 0x7f9b6eb4eea2 process::ProcessManager::resume()
> @ 0x7f9b6eb52bb6 
> _ZNSt6thread11_State_implISt12_Bind_simpleIFZN7process14ProcessManager12init_threadsEvEUlvE_vEEE6_M_runEv
> @ 0x7f9b6bcb283f (unknown)
> @ 0x7f9b6b7c96da start_thread
> @ 0x7f9b6b503d7f (unknown)
> {noformat}
> Attached logs of 3 crash instances.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-8699) mesos agent id is empty in agent HTTP endpoint /state or /state.json

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-8699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407053#comment-16407053
 ] 

Joseph Wu commented on MESOS-8699:
--

[~pwzgorilla] Has this particular agent registered with a master?  The AgentID 
is supplied by the master at registration time (for new agents).

> mesos agent id is empty in agent HTTP endpoint /state or /state.json
> 
>
> Key: MESOS-8699
> URL: https://issues.apache.org/jira/browse/MESOS-8699
> Project: Mesos
>  Issue Type: Bug
>  Components: agent
>Reporter: pwzgorilla
>Priority: Minor
>
> when get data from /state or /state.json http endpoint on agent, agent id is 
> empty:
>  
> "frameworks": [],
>     "git_sha": "f7e3872b0359c6095f8eeaefe408cb7dcef5bb83",
>     "git_tag": "1.5.0",
>     "hostname": "192.168.99.105",
>     {color:#FF}*"id": "",*{color}
>     "log_dir": "/var/log/mesos",
>     "pid": "slave(1)@127.0.0.1:5051",
>     "reserved_resources": {},
>     "reserved_resources_allocated": {},
>     "reserved_resources_full": {},
>     "resources": {
>         "cpus": 1.0,
>         "disk": 12274.0,
>         "gpus": 0.0,
>         "mem": 496.0,
>         "ports": "[31000-32000]"
>     },



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (MESOS-8645) Scheduler HTTP API calls for Accepting/Declining inverse offers lacks some documentation

[Joseph Wu (JIRA)]

Joseph Wu created MESOS-8645:


 Summary: Scheduler HTTP API calls for Accepting/Declining inverse 
offers lacks some documentation
 Key: MESOS-8645
 URL: https://issues.apache.org/jira/browse/MESOS-8645
 Project: Mesos
  Issue Type: Documentation
  Components: documentation
Reporter: Joseph Wu


This issue refers to documentation like:
https://github.com/apache/mesos/blob/master/docs/scheduler-http-api.md#accept

(The .proto file includes in-line documentation about the inverse offer calls.)

The markdown version of HTTP API documentation typically includes an example 
call body and some more details about the implications of the call(s); and 
recommendations of each call(s)' usage.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-8623) Crashed framework brings down the whole Mesos cluster

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16380785#comment-16380785
 ] 

Joseph Wu commented on MESOS-8623:
--

Do you have more master logs prior to the crash?  A full run (from master 
starting to master crashing) would probably help track down which code path is 
unguarded.

BTW, the precise framework probably does not matter too much.  

> Crashed framework brings down the whole Mesos cluster
> -
>
> Key: MESOS-8623
> URL: https://issues.apache.org/jira/browse/MESOS-8623
> Project: Mesos
>  Issue Type: Bug
>  Components: master
>Affects Versions: 1.4.1
> Environment: Debian 8
> Mesos 1.4.1
>Reporter: Tomas Barton
>Priority: Critical
>
> It might be hard to replicate, but when you do your Mesos cluster is gone. 
> The issue was caused by an unresponsive Docker engine on a single agent node. 
> Unfortunately even after fixing Docker issues, all Mesos masters repeatedly 
> failed to start. In despair I've deleted all {{replicated_log}} data from 
> Master and ZooKeeper. Even after that messages agent's {{replicated_log}} got 
> replayed and the master crashed again. Average lifetime for Mesos master was 
> less than 1min.
> {code}
> mesos-master[3814]: I0228 00:25:55.269835  3828 network.hpp:436] ZooKeeper 
> group memberships changed
> mesos-master[3814]: I0228 00:25:55.269979  3832 group.cpp:700] Trying to get 
> '/mesos/log_replicas/002519' in ZooKeeper
> mesos-master[3814]: I0228 00:25:55.271117  3832 group.cpp:700] Trying to get 
> '/mesos/log_replicas/002520' in ZooKeeper
> mesos-master[3814]: I0228 00:25:55.277971  3832 group.cpp:700] Trying to get 
> '/mesos/log_replicas/002521' in ZooKeeper
> mesos-master[3814]: I0228 00:25:55.279296  3827 network.hpp:484] ZooKeeper 
> group PIDs: { log-replica(1)
> mesos-master[3814]: W0228 00:26:15.261255  3831 master.hpp:2372] Master 
> attempted to send message to disconnected framework 
> 911c4b47-2ba7-4959-b59e-c48d896fe210-0005 (kafka)
> mesos-master[3814]: F0228 00:26:15.261318  3831 master.hpp:2382] 
> CHECK_SOME(pid): is NONE
> mesos-master[3814]: *** Check failure stack trace: ***
> mesos-master[3814]: @ 0x7f7187ca073d  google::LogMessage::Fail()
> mesos-master[3814]: @ 0x7f7187ca23bd  google::LogMessage::SendToLog()
> mesos-master[3814]: @ 0x7f7187ca0302  google::LogMessage::Flush()
> mesos-master[3814]: @ 0x7f7187ca2da9  
> google::LogMessageFatal::~LogMessageFatal()
> mesos-master[3814]: @ 0x7f7186d6d769  _CheckFatal::~_CheckFatal()
> mesos-master[3814]: @ 0x7f71870465d5  
> mesos::internal::master::Framework::send<>()
> mesos-master[3814]: @ 0x7f7186fcfe8a  
> mesos::internal::master::Master::executorMessage()
> mesos-master[3814]: @ 0x7f718706b1a1  ProtobufProcess<>::handler4<>()
> mesos-master[3814]: @ 0x7f7187008e36  
> std::_Function_handler<>::_M_invoke()
> mesos-master[3814]: @ 0x7f71870293d1  ProtobufProcess<>::visit()
> mesos-master[3814]: @ 0x7f7186fb7ee4  
> mesos::internal::master::Master::_visit()
> mesos-master[3814]: @ 0x7f7186fd0d5d  
> mesos::internal::master::Master::visit()
> mesos-master[3814]: @ 0x7f7187c02e22  process::ProcessManager::resume()
> mesos-master[3814]: @ 0x7f7187c08d46  
> _ZNSt6thread5_ImplISt12_Bind_simpleIFZN7process14ProcessManager12init_threadsEvEUlvE_vE
> mesos-master[3814]: @ 0x7f7185babca0  (unknown)
> mesos-master[3814]: @ 0x7f71853c6064  start_thread
> mesos-master[3814]: @ 0x7f71850fb62d  (unknown)
> systemd[1]: mesos-master.service: main process exited, code=killed, 
> status=6/ABRT
> systemd[1]: Unit mesos-master.service entered failed state.
> systemd[1]: mesos-master.service holdoff time over, scheduling restart.
> systemd[1]: Stopping Mesos Master...
> systemd[1]: Starting Mesos Master...
> systemd[1]: Started Mesos Master.
> mesos-master[27840]: WARNING: Logging before InitGoogleLogging() is written 
> to STDERR
> mesos-master[27840]: I0228 01:32:38.294122 27829 main.cpp:232] Build: 
> 2017-11-18 02:15:41 by admin
> mesos-master[27840]: I0228 01:32:38.294168 27829 main.cpp:233] Version: 1.4.1
> mesos-master[27840]: I0228 01:32:38.294178 27829 main.cpp:236] Git tag: 1.4.1
> mesos-master[27840]: I0228 01:32:38.294186 27829 main.cpp:240] Git SHA: 
> c844db9ac7c0cef59be87438c6781bfb71adcc42
> mesos-master[27840]: I0228 01:32:38.296067 27829 main.cpp:340] Using 
> 'HierarchicalDRF' allocator
> mesos-master[27840]: I0228 01:32:38.411576 27829 replica.cpp:779] Replica 
> recovered with log positions 13 -> 14 with 0 holes and 0 unlearned
> mesos-master[27840]: 2018-02-28 
> 01:32:38,412:27829(0x7fab44755700):ZOO_INFO@log_env@726: Client 
> environment:zookeeper.version=zookeeper C client 3.4.8
> mesos-master[27840]: 2018-02-28 
> 

[jira] [Commented] (MESOS-8623) Crashed framework brings down the whole Mesos cluster

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16380688#comment-16380688
 ] 

Joseph Wu commented on MESOS-8623:
--

This CHECK can be hit whenever the master attempts to send any message to a 
framework recovered via Agent registration (i.e. Agent reports that the 
framework is running tasks on it), but before the framework has reconnected to 
the master.  

The master should be guarding against sending messages to disconnected 
frameworks, so we'll have to track down which code path is responsible for 
sending this message.

A cursory {{git blame}} suggests that this crash could have happened from 1.2.0 
onwards, but this also depends on which message is being sent:
https://github.com/apache/mesos/commit/0dbceafa3b7caba9e541cd64bce3d5421e3b1262

Note: This commit removed a {{return;}} that would have hidden this bug.
https://github.com/apache/mesos/commit/65efb347301f90e638361a50282cb74980c4c081

> Crashed framework brings down the whole Mesos cluster
> -
>
> Key: MESOS-8623
> URL: https://issues.apache.org/jira/browse/MESOS-8623
> Project: Mesos
>  Issue Type: Bug
>  Components: master
>Affects Versions: 1.4.1
> Environment: Debian 8
> Mesos 1.4.1
>Reporter: Tomas Barton
>Priority: Critical
>
> It might be hard to replicate, but when you do your Mesos cluster is gone. 
> The issue was caused by an unresponsive Docker engine on a single agent node. 
> Unfortunately even after fixing Docker issues, all Mesos masters repeatedly 
> failed to start. In despair I've deleted all {{replicated_log}} data from 
> Master and ZooKeeper. Even after that messages agent's {{replicated_log}} got 
> replayed and the master crashed again. Average lifetime for Mesos master was 
> less than 1min.
> {code}
> mesos-master[3814]: I0228 00:25:55.269835  3828 network.hpp:436] ZooKeeper 
> group memberships changed
> mesos-master[3814]: I0228 00:25:55.269979  3832 group.cpp:700] Trying to get 
> '/mesos/log_replicas/002519' in ZooKeeper
> mesos-master[3814]: I0228 00:25:55.271117  3832 group.cpp:700] Trying to get 
> '/mesos/log_replicas/002520' in ZooKeeper
> mesos-master[3814]: I0228 00:25:55.277971  3832 group.cpp:700] Trying to get 
> '/mesos/log_replicas/002521' in ZooKeeper
> mesos-master[3814]: I0228 00:25:55.279296  3827 network.hpp:484] ZooKeeper 
> group PIDs: { log-replica(1)
> mesos-master[3814]: W0228 00:26:15.261255  3831 master.hpp:2372] Master 
> attempted to send message to disconnected framework 
> 911c4b47-2ba7-4959-b59e-c48d896fe210-0005 (kafka)
> mesos-master[3814]: F0228 00:26:15.261318  3831 master.hpp:2382] 
> CHECK_SOME(pid): is NONE
> mesos-master[3814]: *** Check failure stack trace: ***
> mesos-master[3814]: @ 0x7f7187ca073d  google::LogMessage::Fail()
> mesos-master[3814]: @ 0x7f7187ca23bd  google::LogMessage::SendToLog()
> mesos-master[3814]: @ 0x7f7187ca0302  google::LogMessage::Flush()
> mesos-master[3814]: @ 0x7f7187ca2da9  
> google::LogMessageFatal::~LogMessageFatal()
> mesos-master[3814]: @ 0x7f7186d6d769  _CheckFatal::~_CheckFatal()
> mesos-master[3814]: @ 0x7f71870465d5  
> mesos::internal::master::Framework::send<>()
> mesos-master[3814]: @ 0x7f7186fcfe8a  
> mesos::internal::master::Master::executorMessage()
> mesos-master[3814]: @ 0x7f718706b1a1  ProtobufProcess<>::handler4<>()
> mesos-master[3814]: @ 0x7f7187008e36  
> std::_Function_handler<>::_M_invoke()
> mesos-master[3814]: @ 0x7f71870293d1  ProtobufProcess<>::visit()
> mesos-master[3814]: @ 0x7f7186fb7ee4  
> mesos::internal::master::Master::_visit()
> mesos-master[3814]: @ 0x7f7186fd0d5d  
> mesos::internal::master::Master::visit()
> mesos-master[3814]: @ 0x7f7187c02e22  process::ProcessManager::resume()
> mesos-master[3814]: @ 0x7f7187c08d46  
> _ZNSt6thread5_ImplISt12_Bind_simpleIFZN7process14ProcessManager12init_threadsEvEUlvE_vE
> mesos-master[3814]: @ 0x7f7185babca0  (unknown)
> mesos-master[3814]: @ 0x7f71853c6064  start_thread
> mesos-master[3814]: @ 0x7f71850fb62d  (unknown)
> systemd[1]: mesos-master.service: main process exited, code=killed, 
> status=6/ABRT
> systemd[1]: Unit mesos-master.service entered failed state.
> systemd[1]: mesos-master.service holdoff time over, scheduling restart.
> systemd[1]: Stopping Mesos Master...
> systemd[1]: Starting Mesos Master...
> systemd[1]: Started Mesos Master.
> mesos-master[27840]: WARNING: Logging before InitGoogleLogging() is written 
> to STDERR
> mesos-master[27840]: I0228 01:32:38.294122 27829 main.cpp:232] Build: 
> 2017-11-18 02:15:41 by admin
> mesos-master[27840]: I0228 01:32:38.294168 27829 main.cpp:233] Version: 1.4.1
> mesos-master[27840]: I0228 01:32:38.294178 27829 main.cpp:236] Git tag: 1.4.1
> mesos-master[27840]: I0228 

[jira] [Commented] (MESOS-3676) Port process/collect.hpp to Windows

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359210#comment-16359210
 ] 

Joseph Wu commented on MESOS-3676:
--

The tests didn't actually exist prior to your commit.  And there was nothing to 
fix about the tests after they were added :)

> Port process/collect.hpp to Windows
> ---
>
> Key: MESOS-3676
> URL: https://issues.apache.org/jira/browse/MESOS-3676
> Project: Mesos
>  Issue Type: Task
>  Components: libprocess
>Reporter: Alex Clemmer
>Assignee: Benjamin Mahler
>Priority: Major
>  Labels: mesosphere, windows
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-3382) Include libsvn in Windows CMake build

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359071#comment-16359071
 ] 

Joseph Wu commented on MESOS-3382:
--

We don't need the SVN library because of the following dependency chain:
1) MESOS-5932 - LevelDB isn't supported on Windows yet.
2) No LevelDB means we can't enabled the replicated log (i.e. for multiple 
masters on Windows).
3) The replicated log is the only code that uses libsvn at the moment.

> Include libsvn in Windows CMake build
> -
>
> Key: MESOS-3382
> URL: https://issues.apache.org/jira/browse/MESOS-3382
> Project: Mesos
>  Issue Type: Task
>  Components: cmake
>Reporter: Alex Clemmer
>Priority: Minor
>  Labels: build, cmake, mesosphere, windows
>
> Windows will probably require libsvn to work. This means we need to insert 
> the code to retrieve, build, and link against it for the Windows path, since 
> it isn't rebundled and distributed as part of Mesos.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (MESOS-5231) Create Design Doc for Manage offers in allocator

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu reassigned MESOS-5231:


Assignee: Joseph Wu

> Create Design Doc for Manage offers in allocator
> 
>
> Key: MESOS-5231
> URL: https://issues.apache.org/jira/browse/MESOS-5231
> Project: Mesos
>  Issue Type: Bug
>Reporter: Klaus Ma
>Assignee: Joseph Wu
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (MESOS-4553) Manage offers in allocator.

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu reassigned MESOS-4553:


Assignee: Joseph Wu

> Manage offers in allocator.
> ---
>
> Key: MESOS-4553
> URL: https://issues.apache.org/jira/browse/MESOS-4553
> Project: Mesos
>  Issue Type: Epic
>  Components: master
>Reporter: Klaus Ma
>Assignee: Joseph Wu
>Priority: Major
>
> Currently, the {{offers}} are managed by {{Master}} which introduces two 
> issues:
> 1. In Quota, master rescind more offers to address race condition
> 2. Allocator can not modify offers: resources return to allocator and offer 
> again,  that impact resources utilisation & performance (MESOS-3078)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (MESOS-8555) Support UCR on Windows

[Joseph Wu (JIRA)]

Joseph Wu created MESOS-8555:


 Summary: Support UCR on Windows
 Key: MESOS-8555
 URL: https://issues.apache.org/jira/browse/MESOS-8555
 Project: Mesos
  Issue Type: Epic
  Components: containerization
Reporter: Joseph Wu


Docker container support on Windows relies on calling this shim:
https://github.com/Microsoft/hcsshim

If we want to support container images in the Mesos Containerizer on Windows, 
we may need to look into doing the same thing (or similar).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-6706) Port `files_tests.cpp`

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357598#comment-16357598
 ] 

Joseph Wu commented on MESOS-6706:
--

Two remaining disabled tests in this file:
{code}
TEST_F_TEMP_DISABLED_ON_WINDOWS(FilesTest, ResolveTest)
TEST_F_TEMP_DISABLED_ON_WINDOWS(FilesTest, BrowseTest)
{code}

> Port `files_tests.cpp`
> --
>
> Key: MESOS-6706
> URL: https://issues.apache.org/jira/browse/MESOS-6706
> Project: Mesos
>  Issue Type: Bug
>  Components: agent
>Reporter: Alex Clemmer
>Assignee: Andrew Schwartzmeyer
>Priority: Major
>  Labels: microsoft, windows-mvp
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-7450) Docker containerizer will leak dangling symlinks if restarted with a colon in the sandbox path

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357440#comment-16357440
 ] 

Joseph Wu commented on MESOS-7450:
--

Yes.  Here's a relevant TODO:
https://github.com/apache/mesos/blob/c665dd6c22715fa941200020a8f7209f1f5b1ca1/src/slave/containerizer/docker.hpp#L445-L448

> Docker containerizer will leak dangling symlinks if restarted with a colon in 
> the sandbox path
> --
>
> Key: MESOS-7450
> URL: https://issues.apache.org/jira/browse/MESOS-7450
> Project: Mesos
>  Issue Type: Bug
>  Components: containerization, docker
>Affects Versions: 0.21.0, 1.2.0
>Reporter: Joseph Wu
>Priority: Major
>  Labels: mesosphere
>
> The Docker CLI has a limitation, which was worked around in MESOS-1833.
> TL;DR: If you launch a container with a colon ({{:}}) in the sandbox path, we 
> will create a symlink to that path and mount that symlink into the Docker 
> container.
> However, when you restart the Mesos agent after launching a container like 
> the above, the Docker containerizer will "forget" about the symlink and 
> thereby not clean it up when the container exits.  We will still GC the 
> actual sandbox, but not the symlink.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-2487) Ensure protobuf "==" operator does not go out of sync with new protobuf fields

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354905#comment-16354905
 ] 

Joseph Wu commented on MESOS-2487:
--

Playing around with what the code might look like with this class:
https://reviews.apache.org/r/65541/

> Ensure protobuf "==" operator does not go out of sync with new protobuf fields
> --
>
> Key: MESOS-2487
> URL: https://issues.apache.org/jira/browse/MESOS-2487
> Project: Mesos
>  Issue Type: Task
>Reporter: Vinod Kone
>Priority: Major
>
> Currently when a new field is added to a protobuf that has a custom "==" 
> operator defined,  we don't make sure that the field is accounted for in the 
> comparison. Ideally we should catch such errors at build time or 'make check' 
> time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (MESOS-8510) URI disk profile adaptor does not consider plugin type for a profile.

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-8510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-8510:
-
Story Points: 3

> URI disk profile adaptor does not consider plugin type for a profile.
> -
>
> Key: MESOS-8510
> URL: https://issues.apache.org/jira/browse/MESOS-8510
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 1.5.0
>Reporter: Jie Yu
>Assignee: Joseph Wu
>Priority: Major
>
> Currently, the URI disk profile adaptor will fetch an URI, the content of 
> which contains a profile matrix. However, there's no field in the profile 
> matrix for the adaptor to tell which plugin type a profile is for.
> We should consider adding a `plugin_type` field in `CSIManifest`.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (MESOS-7882) Mesos master rescinds all the in-flight offers from all the registered agents when a new maintenance schedule is posted for a subset of slaves

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-7882:
-
Shepherd: Benjamin Mahler
Story Points: 3
  Labels: maintenance mesosphere  (was: )
  Sprint: Mesosphere Sprint 73
Target Version/s: 1.6.0

> Mesos master rescinds all the in-flight offers from all the registered agents 
> when a new maintenance schedule is posted for a subset of slaves
> --
>
> Key: MESOS-7882
> URL: https://issues.apache.org/jira/browse/MESOS-7882
> Project: Mesos
>  Issue Type: Bug
>  Components: master
>Affects Versions: 1.3.0
> Environment: Ubuntu 14:04(trusty)
> Mesos master branch.
> SHA: a31dd52ab71d2a529b55cd9111ec54acf7550ded
>Reporter: Sagar Sadashiv Patwardhan
>Assignee: Joseph Wu
>Priority: Minor
>  Labels: maintenance, mesosphere
>
> We are running mesos 1.1.0 in production. We use a custom autoscaler for 
> scaling our mesos  cluster up and down. While scaling down the cluster, 
> autoscaler makes a POST request to mesos master /maintenance/schedule 
> endpoint with a set of slaves to move to maintenance mode. This forces mesos 
> master to rescind all the in-flight offers from *all the slaves* in the 
> cluster. If our scheduler accepts one of these offers, then we get a 
> TASK_LOST status update back for that task. We also see such 
> (https://gist.github.com/sagar8192/8858e7cb59a23e8e1762a27571824118) log 
> lines in mesos master logs.
> After reading the code(refs: 
> https://github.com/apache/mesos/blob/master/src/master/master.cpp#L6772), it 
> appears that offers are getting rescinded for all the slaves. I am not sure 
> what is the expected behavior here, but it makes more sense if only resources 
> from slaves marked for maintenance are reclaimed.
> *Experiment:*
> To verify that it is actually happening, I checked out the master branch(sha: 
> a31dd52ab71d2a529b55cd9111ec54acf7550ded ) and added some log 
> lines(https://gist.github.com/sagar8192/42ca055720549c5ff3067b1e6c7c68b3). 
> Built the binary and started a mesos master and 2 agent processes. Used a 
> basic python framework that launches docker containers on these slaves. 
> Verified that there is no existing schedule for any slaves using `curl 
> 10.40.19.239:5050/maintenance/status`. Posted maintenance schedule for one of 
> the 
> slaves(https://gist.github.com/sagar8192/fb65170240dd32a53f27e6985c549df0) 
> after starting the mesos framework.
> *Logs:*
> mesos-master: 
> https://gist.github.com/sagar8192/91888419fdf8284e33ebd58351131203
> mesos-slave1: 
> https://gist.github.com/sagar8192/3a83364b1f5ffc63902a80c728647f31
> mesos-slave2: 
> https://gist.github.com/sagar8192/1b341ef2271dde11d276974a27109426
> Mesos framework: 
> https://gist.github.com/sagar8192/bcd4b37dba03bde0a942b5b972004e8a
> I think mesos should rescind offers and inverse offers only for those slaves 
> that are marked for maintenance(draining mode).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (MESOS-7882) Mesos master rescinds all the in-flight offers from all the registered agents when a new maintenance schedule is posted for a subset of slaves

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu reassigned MESOS-7882:


Assignee: Joseph Wu

> Mesos master rescinds all the in-flight offers from all the registered agents 
> when a new maintenance schedule is posted for a subset of slaves
> --
>
> Key: MESOS-7882
> URL: https://issues.apache.org/jira/browse/MESOS-7882
> Project: Mesos
>  Issue Type: Bug
>  Components: master
>Affects Versions: 1.3.0
> Environment: Ubuntu 14:04(trusty)
> Mesos master branch.
> SHA: a31dd52ab71d2a529b55cd9111ec54acf7550ded
>Reporter: Sagar Sadashiv Patwardhan
>Assignee: Joseph Wu
>Priority: Minor
>
> We are running mesos 1.1.0 in production. We use a custom autoscaler for 
> scaling our mesos  cluster up and down. While scaling down the cluster, 
> autoscaler makes a POST request to mesos master /maintenance/schedule 
> endpoint with a set of slaves to move to maintenance mode. This forces mesos 
> master to rescind all the in-flight offers from *all the slaves* in the 
> cluster. If our scheduler accepts one of these offers, then we get a 
> TASK_LOST status update back for that task. We also see such 
> (https://gist.github.com/sagar8192/8858e7cb59a23e8e1762a27571824118) log 
> lines in mesos master logs.
> After reading the code(refs: 
> https://github.com/apache/mesos/blob/master/src/master/master.cpp#L6772), it 
> appears that offers are getting rescinded for all the slaves. I am not sure 
> what is the expected behavior here, but it makes more sense if only resources 
> from slaves marked for maintenance are reclaimed.
> *Experiment:*
> To verify that it is actually happening, I checked out the master branch(sha: 
> a31dd52ab71d2a529b55cd9111ec54acf7550ded ) and added some log 
> lines(https://gist.github.com/sagar8192/42ca055720549c5ff3067b1e6c7c68b3). 
> Built the binary and started a mesos master and 2 agent processes. Used a 
> basic python framework that launches docker containers on these slaves. 
> Verified that there is no existing schedule for any slaves using `curl 
> 10.40.19.239:5050/maintenance/status`. Posted maintenance schedule for one of 
> the 
> slaves(https://gist.github.com/sagar8192/fb65170240dd32a53f27e6985c549df0) 
> after starting the mesos framework.
> *Logs:*
> mesos-master: 
> https://gist.github.com/sagar8192/91888419fdf8284e33ebd58351131203
> mesos-slave1: 
> https://gist.github.com/sagar8192/3a83364b1f5ffc63902a80c728647f31
> mesos-slave2: 
> https://gist.github.com/sagar8192/1b341ef2271dde11d276974a27109426
> Mesos framework: 
> https://gist.github.com/sagar8192/bcd4b37dba03bde0a942b5b972004e8a
> I think mesos should rescind offers and inverse offers only for those slaves 
> that are marked for maintenance(draining mode).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (MESOS-6705) Port `fetcher_tests.cpp`

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-6705:
-
Fix Version/s: 1.5.0

> Port `fetcher_tests.cpp`
> 
>
> Key: MESOS-6705
> URL: https://issues.apache.org/jira/browse/MESOS-6705
> Project: Mesos
>  Issue Type: Bug
>  Components: agent
>Reporter: Alex Clemmer
>Assignee: Jeff Coffler
>Priority: Major
>  Labels: microsoft, windows-mvp
> Fix For: 1.5.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (MESOS-6709) Enable HTTP and TCP health checks on Windows

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-6709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-6709:
-
Fix Version/s: 1.5.0

> Enable HTTP and TCP health checks on Windows
> 
>
> Key: MESOS-6709
> URL: https://issues.apache.org/jira/browse/MESOS-6709
> Project: Mesos
>  Issue Type: Task
>  Components: agent
>Reporter: Alex Clemmer
>Assignee: John Kordich
>Priority: Blocker
>  Labels: microsoft, windows, windows-mvp
> Fix For: 1.5.0
>
>
> This specifically excludes command health checks, as these are blocked on the 
> IO Switchboard code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-8278) Mesos Containerizer cannot recover due to check failure.

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292980#comment-16292980
 ] 

Joseph Wu commented on MESOS-8278:
--

[~jieyu] Is this fixed by https://reviews.apache.org/r/64623/ ?

> Mesos Containerizer cannot recover due to check failure.
> 
>
> Key: MESOS-8278
> URL: https://issues.apache.org/jira/browse/MESOS-8278
> Project: Mesos
>  Issue Type: Bug
>  Components: containerization
>Reporter: Gilbert Song
>Priority: Critical
>  Labels: containerizer, csi-post-mvp, standalone
>
> Mesos containerizer cannot recover due to a check failure on nested 
> container's sandbox directory.
> {noformat}
> I1129 22:00:42.556479  5812 containerizer.cpp:670] Recovering containerizer
> F1129 22:00:42.560739  5812 containerizer.cpp:912] CHECK_SOME(directory): is 
> NONE 
> *** Check failure stack trace: ***
> @ 0x7f7e6cf1294d  google::LogMessage::Fail()
> @ 0x7f7e6cf11d1e  google::LogMessage::SendToLog()
> @ 0x7f7e6cf1261d  google::LogMessage::Flush()
> @ 0x7f7e6cf15a98  google::LogMessageFatal::~LogMessageFatal()
> @ 0x55ca72a95197  _CheckFatal::~_CheckFatal()
> @ 0x7f7e6bb23770  
> mesos::internal::slave::MesosContainerizerProcess::recover()
> @ 0x7f7e6bbe643c  
> _ZZN7process8dispatchI7NothingN5mesos8internal5slave25MesosContainerizerProcessERK6OptionINS4_5state10SlaveStateEESB_EENS_6FutureIT_EERKNS_3PIDIT0_EEMSG_FSE_T1_EOT2_ENKUlRS9_PNS_11ProcessBaseEE_clESP_SR_
> @ 0x7f7e6bbe6295  
> _ZNSt5_BindIFZN7process8dispatchI7NothingN5mesos8internal5slave25MesosContainerizerProcessERK6OptionINS5_5state10SlaveStateEESC_EENS0_6FutureIT_EERKNS0_3PIDIT0_EEMSH_FSF_T1_EOT2_EUlRSA_PNS0_11ProcessBaseEE_SA_St12_PlaceholderILi16__callIvJOSS_EJLm0ELm1SE_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
> @ 0x7f7e6bbe61f6  
> _ZNSt5_BindIFZN7process8dispatchI7NothingN5mesos8internal5slave25MesosContainerizerProcessERK6OptionINS5_5state10SlaveStateEESC_EENS0_6FutureIT_EERKNS0_3PIDIT0_EEMSH_FSF_T1_EOT2_EUlRSA_PNS0_11ProcessBaseEE_SA_St12_PlaceholderILi1clIJSS_EvEESH_DpOT_
> @ 0x7f7e6bbe5f02  
> _ZNSt17_Function_handlerIFvPN7process11ProcessBaseEESt5_BindIFZNS0_8dispatchI7NothingN5mesos8internal5slave25MesosContainerizerProcessERK6OptionINS9_5state10SlaveStateEESG_EENS0_6FutureIT_EERKNS0_3PIDIT0_EEMSL_FSJ_T1_EOT2_EUlRSE_S2_E_SE_St12_PlaceholderILi1E9_M_invokeERKSt9_Any_dataOS2_
> @ 0x7f7e6ce37cf4  std::function<>::operator()()
> @ 0x7f7e6ce1ded4  process::ProcessBase::visit()
> @ 0x7f7e6cea38fe  process::DispatchEvent::visit()
> @ 0x7f7e6a9741b1  process::ProcessBase::serve()
> @ 0x7f7e6ce1a8eb  process::ProcessManager::resume()
> @ 0x7f7e6ce2b86e  
> process::ProcessManager::init_threads()::$_7::operator()()
> @ 0x7f7e6ce2b715  
> _ZNSt12_Bind_simpleIFZN7process14ProcessManager12init_threadsEvE3$_7vEE9_M_invokeIJEEEvSt12_Index_tupleIJXspT_EEE
> @ 0x7f7e6ce2b6e5  std::_Bind_simple<>::operator()()
> @ 0x7f7e6ce2b6bc  std::thread::_Impl<>::_M_run()
> @ 0x7f7e6617d030  (unknown)
> @ 0x7f7e65c966aa  start_thread
> @ 0x7f7e659cbe9d  (unknown)
> {noformat}
> Maybe related to the change of standalone container support.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7303) Support Isolator capabilities.

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16282117#comment-16282117
 ] 

Joseph Wu commented on MESOS-7303:
--

{{1.5.0}} (as you added) sounds good.

> Support Isolator capabilities.
> --
>
> Key: MESOS-7303
> URL: https://issues.apache.org/jira/browse/MESOS-7303
> Project: Mesos
>  Issue Type: Task
>  Components: containerization
>Reporter: Jie Yu
>Assignee: Joseph Wu
>  Labels: mesosphere, storage
>
> Currently, isolators have one capability: whether it supports nesting or not. 
> To support launching containers that are not tied to Mesos tasks or executors 
> (standalone containers), we need to add another capability to the Isolator 
> interface so that we can avoid invoking those isolators that are not yet 
> support that when launching standalone containers.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (MESOS-8251) Introduce a way to resolve the "profile" for disk resources

[Joseph Wu (JIRA)]

Joseph Wu created MESOS-8251:


 Summary: Introduce a way to resolve the "profile" for disk 
resources
 Key: MESOS-8251
 URL: https://issues.apache.org/jira/browse/MESOS-8251
 Project: Mesos
  Issue Type: Task
Reporter: Joseph Wu
Assignee: Joseph Wu


This builds off MESOS-8060 which introduced the {{profile}} field to disk 
resources.

A volume profile will consist of a {{string}} which maps to a list of 
{{VolumeCapability}} and some free-form parameters (string key-value pairs).  
See the {{message CreateVolumeResponse}} under 
https://github.com/container-storage-interface/spec/blob/master/csi.proto for 
more information about the fields.

We will introduce a module which will allow the operator to specify this 
mapping.  Unfortunately, due to the nature of volume profiles, we cannot 
provide a reasonable "default" mapping of profiles (because we don't have any 
idea what volumes are available).  Instead, there will be two implementations 
of the module:
1) The default will essentially turn volume profiles off.  Any attempt to use 
volume profiles will simply fail.
2) An optional module will allow the operator to specify a URI (file or link) 
that contains a JSON representation of the mapping.  The module will have some 
knobs to tune the frequency at which this URI is fetched from.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7882) Mesos master rescinds all the in-flight offers from all the registered agents when a new maintenance schedule is posted for a subset of slaves

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254487#comment-16254487
 ] 

Joseph Wu commented on MESOS-7882:
--

Yeah, this could be improved.  The first iteration of the 
{{/maintenance/schedule}} API precisely updates every machine.  And we never 
got any requests to make this more efficient (until now I suppose :) ).

Note that in the repro by [~huadongliu], the {{MachineID}} is missing a 
hostname, so it does not match either of the two agents.  Both get their 
unavailability "removed" (even though there was no unavailability to remove, 
technically).

> Mesos master rescinds all the in-flight offers from all the registered agents 
> when a new maintenance schedule is posted for a subset of slaves
> --
>
> Key: MESOS-7882
> URL: https://issues.apache.org/jira/browse/MESOS-7882
> Project: Mesos
>  Issue Type: Bug
>  Components: master
>Affects Versions: 1.3.0
> Environment: Ubuntu 14:04(trusty)
> Mesos master branch.
> SHA: a31dd52ab71d2a529b55cd9111ec54acf7550ded
>Reporter: Sagar Sadashiv Patwardhan
>Priority: Minor
>
> We are running mesos 1.1.0 in production. We use a custom autoscaler for 
> scaling our mesos  cluster up and down. While scaling down the cluster, 
> autoscaler makes a POST request to mesos master /maintenance/schedule 
> endpoint with a set of slaves to move to maintenance mode. This forces mesos 
> master to rescind all the in-flight offers from *all the slaves* in the 
> cluster. If our scheduler accepts one of these offers, then we get a 
> TASK_LOST status update back for that task. We also see such 
> (https://gist.github.com/sagar8192/8858e7cb59a23e8e1762a27571824118) log 
> lines in mesos master logs.
> After reading the code(refs: 
> https://github.com/apache/mesos/blob/master/src/master/master.cpp#L6772), it 
> appears that offers are getting rescinded for all the slaves. I am not sure 
> what is the expected behavior here, but it makes more sense if only resources 
> from slaves marked for maintenance are reclaimed.
> *Experiment:*
> To verify that it is actually happening, I checked out the master branch(sha: 
> a31dd52ab71d2a529b55cd9111ec54acf7550ded ) and added some log 
> lines(https://gist.github.com/sagar8192/42ca055720549c5ff3067b1e6c7c68b3). 
> Built the binary and started a mesos master and 2 agent processes. Used a 
> basic python framework that launches docker containers on these slaves. 
> Verified that there is no existing schedule for any slaves using `curl 
> 10.40.19.239:5050/maintenance/status`. Posted maintenance schedule for one of 
> the 
> slaves(https://gist.github.com/sagar8192/fb65170240dd32a53f27e6985c549df0) 
> after starting the mesos framework.
> *Logs:*
> mesos-master: 
> https://gist.github.com/sagar8192/91888419fdf8284e33ebd58351131203
> mesos-slave1: 
> https://gist.github.com/sagar8192/3a83364b1f5ffc63902a80c728647f31
> mesos-slave2: 
> https://gist.github.com/sagar8192/1b341ef2271dde11d276974a27109426
> Mesos framework: 
> https://gist.github.com/sagar8192/bcd4b37dba03bde0a942b5b972004e8a
> I think mesos should rescind offers and inverse offers only for those slaves 
> that are marked for maintenance(draining mode).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-8146) Mesos agent fails containers on restart if containers were started with memory-swap less than memory + 64mb

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-8146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240756#comment-16240756
 ] 

Joseph Wu commented on MESOS-8146:
--

One important thing to note is that specifying arbitrary parameters to the 
DockerContainerizer is not guaranteed to work:
https://github.com/apache/mesos/blob/1.4.x/include/mesos/mesos.proto#L2850-L2854

The error here probably comes from a conflict between the underlying resource 
isolation.  Under the covers, Mesos can resize the container's cpu/memory.  The 
extra parameters you specify breaks the assumption Mesos is making (about how 
Docker works).

> Mesos agent fails containers on restart if containers were started with 
> memory-swap less than memory + 64mb
> ---
>
> Key: MESOS-8146
> URL: https://issues.apache.org/jira/browse/MESOS-8146
> Project: Mesos
>  Issue Type: Bug
>  Components: agent
>Affects Versions: 1.4.0
> Environment: Mesos 1.4.0
> Redhat 7.4
> Marathon 1.4.8
> docker 1.12.6 
> docker api 1.24
>Reporter: Guchakov Nikita
>
> I'd have some strange behaviour with Mesos when trying to disable swap on 
> docker containers. Our mesos version in use is 1.4.0
> When marathon deploys containers with
> ```
> "parameters": [
> {
>   "key": "memory",
>   "value": "1024m"
> },
> {
>   "key": "memory-swap",
>   "value": "1024m"
> }
>   ]
> ```
> then it deploys successfully. BUT when mesos-slave restarts and tries to 
> deregister executor it fails:
> ```E1027 11:11:47.367416 12626 slave.cpp:4287] Failed to update resources for 
> container 6e3e07af-db09-4dc0-88f8-4e5599529cbe of executor 
> 'templates-api.d72549fd-baed-11e7-9742-96b37b4eca54' of framework 
> 20171020-202151-141892780-5050-1-0001, destroying container: Failed to set 
> 'memory.limit_in_bytes': Invalid argument
> ```
> Things goes more weird when I tried different memory-swap configurations:
> Containers doesn't destroyed on slave's restart only when memory-swap >= 
> memory + 64mb.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-8085) No point in deallocate() for a framework for maintenance if it is deactivated.

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-8085:
-
Shepherd: Joseph Wu

> No point in deallocate() for a framework for maintenance if it is deactivated.
> --
>
> Key: MESOS-8085
> URL: https://issues.apache.org/jira/browse/MESOS-8085
> Project: Mesos
>  Issue Type: Bug
>Reporter: Yan Xu
>  Labels: maintenance
>
> The {{UnavailableResources}} sent from the allocator to the master are going 
> to be dropped by the master anyways, which results in the following line to 
> be printed per inactive framework per allocation which spams the master log. 
> We could tune down the log level but it's better to just not send the 
> {{UnavailableResources}} by the allocator.
> {code:title=}
> LOG(INFO) << "Master ignoring inverse offers to framework " << frameworkId
>   << " because the framework has terminated or is inactive";
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7509) CniIsolatorPortMapperTest.ROOT_INTERNET_CURL_PortMapper fails on some Linux distros.

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206154#comment-16206154
 ] 

Joseph Wu commented on MESOS-7509:
--

+1 to disabling the tests on CentOS 6, given we write up the necessary 
precautionary documentation.

> CniIsolatorPortMapperTest.ROOT_INTERNET_CURL_PortMapper fails on some Linux 
> distros.
> 
>
> Key: MESOS-7509
> URL: https://issues.apache.org/jira/browse/MESOS-7509
> Project: Mesos
>  Issue Type: Bug
>  Components: containerization
> Environment: CentOS 6, Ubuntu 12.04
>Reporter: Alexander Rukletsov
>  Labels: containerizer, flaky-test, isolation, mesosphere
> Attachments: ROOT_DOCKER_DefaultDNS-badrun.txt, 
> ROOT_INTERNET_CURL_PortMapper-badrun.txt, 
> ROOT_INTERNET_CURL_PortMapper_failure_log_1.txt, 
> ROOT_INTERNET_CURL_PortMapper_failure_log_2_centos6.txt
>
>
> I see this test failing consistently on CentOS 6 and Ubuntu 12.04.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7935) CMake build should fail immediately for in-source builds

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200575#comment-16200575
 ] 

Joseph Wu commented on MESOS-7935:
--

The two options in Damien's suggestion are undocumented (by CMake), so they are 
considered unsafe for use.

Feel free to post a review with your alternative.

> CMake build should fail immediately for in-source builds
> 
>
> Key: MESOS-7935
> URL: https://issues.apache.org/jira/browse/MESOS-7935
> Project: Mesos
>  Issue Type: Improvement
>  Components: cmake
> Environment: macOS 10.12
> GNU/Linux Debian Stretch
>Reporter: Damien Gerard
>Assignee: Nathan Jackson
>  Labels: build
>
> In-source builds are neither recommended or supported.  It is simple enough 
> to add a check to fail the build immediately.
> ---
> In-source build of master branch was broken with:
> {noformat}
> cd /Users/damien.gerard/projects/acp/mesos/src && 
> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
>   -DBUILD_FLAGS=\"\" -DBUILD_JAVA_JVM_LIBRARY=\"\" -DHAS_AUTHENTICATION=1 
> -DLIBDIR=\"/usr/local/libmesos\" -DPICOJSON_USE_INT64 
> -DPKGDATADIR=\"/usr/local/share/mesos\" 
> -DPKGLIBEXECDIR=\"/usr/local/libexec/mesos\" -DUSE_CMAKE_BUILD_CONFIG 
> -DUSE_STATIC_LIB -DVERSION=\"1.4.0\" -D__STDC_FORMAT_MACROS 
> -Dmesos_1_4_0_EXPORTS -I/Users/damien.gerard/projects/acp/mesos/include 
> -I/Users/damien.gerard/projects/acp/mesos/include/mesos 
> -I/Users/damien.gerard/projects/acp/mesos/src -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/protobuf-3.3.0/src/protobuf-3.3.0-lib/lib/include
>  -isystem /Users/damien.gerard/projects/acp/mesos/3rdparty/libprocess/include 
> -isystem /usr/local/opt/apr/libexec/include/apr-1 -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/boost-1.53.0/src/boost-1.53.0
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/elfio-3.2/src/elfio-3.2 
> -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/glog-0.3.3/src/glog-0.3.3-lib/lib/include
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/nvml-352.79/src/nvml-352.79 
> -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/picojson-1.3.0/src/picojson-1.3.0
>  -isystem /usr/local/include/subversion-1 -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/stout/include -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/http_parser-2.6.2/src/http_parser-2.6.2
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/concurrentqueue-1.0.0-beta/src/concurrentqueue-1.0.0-beta
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/libev-4.22/src/libev-4.22 
> -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/zookeeper-3.4.8/src/zookeeper-3.4.8/src/c/include
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/zookeeper-3.4.8/src/zookeeper-3.4.8/src/c/generated
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/leveldb-1.19/src/leveldb-1.19/include
>   -std=c++11 -fPIC   -o 
> CMakeFiles/mesos-1.4.0.dir/slave/containerizer/mesos/provisioner/backends/copy.cpp.o
>  -c 
> /Users/damien.gerard/projects/acp/mesos/src/slave/containerizer/mesos/provisioner/backends/copy.cpp
> /Users/damien.gerard/projects/acp/mesos/src/slave/containerizer/mesos/provisioner/appc/store.cpp:132:46:
>  error: no member named 'fetcher' in namespace 'mesos::uri'; did you mean 
> 'Fetcher'?
>   Try uriFetcher = uri::fetcher::create();
> ~^~~
>  Fetcher
> /Users/damien.gerard/projects/acp/mesos/include/mesos/uri/fetcher.hpp:46:7: 
> note: 'Fetcher' declared here
> class Fetcher
>   ^
> /Users/damien.gerard/projects/acp/mesos/src/slave/containerizer/mesos/provisioner/appc/store.cpp:132:55:
>  error: no member named 'create' in 'mesos::uri::Fetcher'
>   Try uriFetcher = uri::fetcher::create();
> {noformat}
> Both Linux & macOS, not tested elsewhere, on {{master}} and tag 1.4.0-rc3



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-7935) CMake build should fail immediately for in-source builds

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu reassigned MESOS-7935:


Assignee: Nathan Jackson

> CMake build should fail immediately for in-source builds
> 
>
> Key: MESOS-7935
> URL: https://issues.apache.org/jira/browse/MESOS-7935
> Project: Mesos
>  Issue Type: Improvement
>  Components: cmake
> Environment: macOS 10.12
> GNU/Linux Debian Stretch
>Reporter: Damien Gerard
>Assignee: Nathan Jackson
>  Labels: build
>
> In-source builds are neither recommended or supported.  It is simple enough 
> to add a check to fail the build immediately.
> ---
> In-source build of master branch was broken with:
> {noformat}
> cd /Users/damien.gerard/projects/acp/mesos/src && 
> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
>   -DBUILD_FLAGS=\"\" -DBUILD_JAVA_JVM_LIBRARY=\"\" -DHAS_AUTHENTICATION=1 
> -DLIBDIR=\"/usr/local/libmesos\" -DPICOJSON_USE_INT64 
> -DPKGDATADIR=\"/usr/local/share/mesos\" 
> -DPKGLIBEXECDIR=\"/usr/local/libexec/mesos\" -DUSE_CMAKE_BUILD_CONFIG 
> -DUSE_STATIC_LIB -DVERSION=\"1.4.0\" -D__STDC_FORMAT_MACROS 
> -Dmesos_1_4_0_EXPORTS -I/Users/damien.gerard/projects/acp/mesos/include 
> -I/Users/damien.gerard/projects/acp/mesos/include/mesos 
> -I/Users/damien.gerard/projects/acp/mesos/src -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/protobuf-3.3.0/src/protobuf-3.3.0-lib/lib/include
>  -isystem /Users/damien.gerard/projects/acp/mesos/3rdparty/libprocess/include 
> -isystem /usr/local/opt/apr/libexec/include/apr-1 -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/boost-1.53.0/src/boost-1.53.0
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/elfio-3.2/src/elfio-3.2 
> -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/glog-0.3.3/src/glog-0.3.3-lib/lib/include
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/nvml-352.79/src/nvml-352.79 
> -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/picojson-1.3.0/src/picojson-1.3.0
>  -isystem /usr/local/include/subversion-1 -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/stout/include -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/http_parser-2.6.2/src/http_parser-2.6.2
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/concurrentqueue-1.0.0-beta/src/concurrentqueue-1.0.0-beta
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/libev-4.22/src/libev-4.22 
> -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/zookeeper-3.4.8/src/zookeeper-3.4.8/src/c/include
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/zookeeper-3.4.8/src/zookeeper-3.4.8/src/c/generated
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/leveldb-1.19/src/leveldb-1.19/include
>   -std=c++11 -fPIC   -o 
> CMakeFiles/mesos-1.4.0.dir/slave/containerizer/mesos/provisioner/backends/copy.cpp.o
>  -c 
> /Users/damien.gerard/projects/acp/mesos/src/slave/containerizer/mesos/provisioner/backends/copy.cpp
> /Users/damien.gerard/projects/acp/mesos/src/slave/containerizer/mesos/provisioner/appc/store.cpp:132:46:
>  error: no member named 'fetcher' in namespace 'mesos::uri'; did you mean 
> 'Fetcher'?
>   Try uriFetcher = uri::fetcher::create();
> ~^~~
>  Fetcher
> /Users/damien.gerard/projects/acp/mesos/include/mesos/uri/fetcher.hpp:46:7: 
> note: 'Fetcher' declared here
> class Fetcher
>   ^
> /Users/damien.gerard/projects/acp/mesos/src/slave/containerizer/mesos/provisioner/appc/store.cpp:132:55:
>  error: no member named 'create' in 'mesos::uri::Fetcher'
>   Try uriFetcher = uri::fetcher::create();
> {noformat}
> Both Linux & macOS, not tested elsewhere, on {{master}} and tag 1.4.0-rc3



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-6705) Port `fetcher_tests.cpp`

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189075#comment-16189075
 ] 

Joseph Wu commented on MESOS-6705:
--

Some additional tests that will likely be impacted by this port:
{code}
commit 772c8f554fe19f8c121c57bee97679cde2646fb8
Author: Gaston Kleiman 
Date:   Mon Oct 2 16:27:32 2017 -0700

Windows: Disable some new tests that fetch local URIs.

Mesos currently conflates paths and URIs.  On platforms where the
path separator '\' is not equal to the URI separator '/', tests
that rely on `path` helpers for URIs will naturally fail.

This disables some new tests that fail on Windows for this reason.

The tests here should eventually be fixed along with MESOS-6705.

Review: https://reviews.apache.org/r/62735/
{code}

> Port `fetcher_tests.cpp`
> 
>
> Key: MESOS-6705
> URL: https://issues.apache.org/jira/browse/MESOS-6705
> Project: Mesos
>  Issue Type: Bug
>  Components: agent
>Reporter: Alex Clemmer
>Assignee: Jeff Coffler
>  Labels: microsoft, windows-mvp
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-3107) Define CMake style guide

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188891#comment-16188891
 ] 

Joseph Wu commented on MESOS-3107:
--

{code}
commit 3f15dedb221a6882fd01b172f584527c496d4e1e
Author: Andrew Schwartzmeyer 
Date:   Mon Oct 2 14:12:30 2017 -0700

CMake: Used `TRUE|FALSE` instead of `ON|OFF` consistently.

While these are equivalent values to CMake, we should be consistent.
Also made uses of `option` consistent, and moved `ENABLE_JAVA` to the
3rdparty section.

Review: https://reviews.apache.org/r/62730/
{code}

> Define CMake style guide
> 
>
> Key: MESOS-3107
> URL: https://issues.apache.org/jira/browse/MESOS-3107
> Project: Mesos
>  Issue Type: Task
>  Components: cmake
>Reporter: Alex Clemmer
>Assignee: Andrew Schwartzmeyer
>  Labels: build, cmake
>
> The short story is that it is important to be principled about how the CMake 
> build system is maintained, because there CMake language makes it difficult 
> to statically verify that a configuration is correct. It is not unique in 
> this regard, but (make is arguably even worse) but it is something that's 
> important to make sure we get right.
> The longer story is, CMake's language is dynamically scoped and often has 
> somewhat odd defaults for variable values (_e.g._, IIRC, target names passed 
> to ExternalProject_Add default to "PREFIX" instead of erroring out). This 
> means that it is rare to get a configuration-time error (_i.e._, CMake 
> usually doesn't say something like "hey this variable isn't defined"), and in 
> large projects, this can make it very difficult to know where definitions 
> come from, or whether it's important that one config routine runs before 
> another. Dynamic scoping also makes it particularly easy to write spaghetti 
> code, which is clearly undesirable for something as important as a build 
> system.
> Thus, it is particularly important that we lay down our expectations for how 
> the CMake system is to be structured. This might include:
> * Function naming (_e.g._, making it easy to tell whether a function was 
> defined by us, and where it was defined; so we might say that we want our 
> functions to have an underscore to start, and start with the package the come 
> from, like libprocess, so that we know where to look for the definition.)
> * What assertions we want to check variable values against, so that we can 
> replace subtle errors (_e.g._, a library is accidentally named something 
> silly like "PREFIX.0.0.1") with an obvious ones (_e.g._, "You have failed to 
> define your target name, so CMake has defaulted to 'PREFIX'; please check 
> your configuration routines")
> * Decisions of what goes where. (_e.g._, the most complex parts of the CMake 
> MVPs is in the configuration routines, like `MesosConfigure.cmake`; to curb 
> this, we should have strict rules about what goes in that file vs other 
> files, and how we know what is to be run before what. Part of this should 
> probably be prominent comments explaining the structure of the project, so 
> that people aren't confused!)
> * And so on.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-8033) Use more idiomatic CMake for compiler features

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-8033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188890#comment-16188890
 ] 

Joseph Wu commented on MESOS-8033:
--

{code}
commit bd5e874c22f0d16fc5494213d319065cf9107d0f
Author: Andrew Schwartzmeyer 
Date:   Mon Oct 2 14:33:52 2017 -0700

CMake: Removed `MESOS_CPPFLAGS` variable.

This was a magic variable that was used to add compiler definitions
globally. Instead, global definitions are now added explicitly with
`add_definitions`, and others with `target_compile_definitions`.

Review: https://reviews.apache.org/r/62731/
{code}

> Use more idiomatic CMake for compiler features
> --
>
> Key: MESOS-8033
> URL: https://issues.apache.org/jira/browse/MESOS-8033
> Project: Mesos
>  Issue Type: Improvement
>  Components: cmake
>Reporter: Andrew Schwartzmeyer
>Priority: Minor
>  Labels: cmake
>
> Specifically, we should replace
> {noformat}
>   string(APPEND CMAKE_CXX_FLAGS " -std=c++11")
> {noformat}
> With {{CMAKE_CXX_STANDARD}}, and use [compile feature 
> requirements|https://cmake.org/cmake/help/latest/manual/cmake-compile-features.7.html#compile-feature-requirements].
> And replace
> {noformat}
>   string(APPEND CMAKE_CXX_FLAGS " -Wformat-security")
> {noformat}
> With compile options instead of appending to {{CMAKE_CXX_FLAGS}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-3110) Harden the CMake system-dependency-locating routines

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-3110:
-
Fix Version/s: 1.5.0

> Harden the CMake system-dependency-locating routines
> 
>
> Key: MESOS-3110
> URL: https://issues.apache.org/jira/browse/MESOS-3110
> Project: Mesos
>  Issue Type: Task
>  Components: cmake
>Reporter: Alex Clemmer
>Assignee: John Kordich
>  Labels: build, cmake
> Fix For: 1.5.0
>
>
> Currently the Mesos project has two flavors of dependency: (1) the 
> dependencies we expect are already on the system (_e.g._, apr, libsvn), and 
> (2) the dependencies that are historically bundled with Mesos (_e.g._, glog).
> Dependency type (1) requires solid modules that will locate them on any 
> system: Linux, BSD, or Windows. This would come for free if we were using 
> CMake 3.0, but we're using CMake 2.8 so that Ubuntu users can install it out 
> of the box, instead of upgrading CMake first.
> This is additionally useful for dependency type (2), where we will expect to 
> have to use these routines when we support both the rebundled dependencies in 
> the `3rdparty/` folder, and system installations of those dependencies.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-3384) Include libsasl in Windows CMake build

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-3384:
-
Shepherd: Joseph Wu  (was: Joris Van Remoortere)

> Include libsasl in Windows CMake build
> --
>
> Key: MESOS-3384
> URL: https://issues.apache.org/jira/browse/MESOS-3384
> Project: Mesos
>  Issue Type: Task
>  Components: cmake
>Reporter: Alex Clemmer
>Assignee: John Kordich
>  Labels: build, cmake, mesosphere
> Fix For: 1.5.0
>
>
> Windows will probably require libsasl to work. This means we need to insert 
> the code to retrieve, build, and link against it for the Windows path, since 
> it isn't rebundled and distributed as part of Mesos.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7942) Mesos slave - docker job exits normally but reporting as TASK_FAILED

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155770#comment-16155770
 ] 

Joseph Wu commented on MESOS-7942:
--

That is the exit status of the executor, not the task.  The executor will exit 
normally even if the task fails.

> Mesos slave - docker job exits normally but reporting as TASK_FAILED
> 
>
> Key: MESOS-7942
> URL: https://issues.apache.org/jira/browse/MESOS-7942
> Project: Mesos
>  Issue Type: Bug
>  Components: agent, docker
>Affects Versions: 1.1.0, 1.2.1, 1.3.1
> Environment: Kernel | OS | Snapshot:   
> 3.8.13-98.7.1.el7uek | OL 7.3 | 7-2017.6.4
>Reporter: Baskar Sikkayan
>
> Mesos version - 1.2.1.
> Jobs are being scheduled using Chronos. Docker job is being invoked properly, 
> but still getting TASK_FAILED error even it completes with exit status ZERO.
> Mesos slave logs :-
> {code}
> I0906 04:15:03.31192810 slave.cpp:1785] Launching task 
> 'ct:150467132:0:Job_Task_Test:' for framework 
> 5175f6c9-0617-4145-ab46-3b7e64dc67ea-
> I0906 04:15:03.31458410 paths.cpp:547] Trying to chown ' 
> /mesos-data/slave-2/slaves/f20ab78e-acd3-407a-b1b6-47d67a947eff-S1/frameworks/5175f6c9-0617-4145-ab46-3b7e64dc67ea-/executors/ct:150467132:0:Job_Task_Test:/runs/7cd8bd78-b20d-4db5-8435-4d1420cb1b93'
>  to user 'root'
> I0906 04:15:03.31514010 slave.cpp:6479] Launching executor 
> 'ct:150467132:0:Job_Task_Test:' of framework 
> 5175f6c9-0617-4145-ab46-3b7e64dc67ea- with resources cpus(*)(allocated: 
> *):0.1; mem(*)(allocated: *):32 in work directory ' 
> /mesos-data/slave-2/slaves/f20ab78e-acd3-407a-b1b6-47d67a947eff-S1/frameworks/5175f6c9-0617-4145-ab46-3b7e64dc67ea-/executors/ct:150467132:0:Job_Task_Test:/runs/7cd8bd78-b20d-4db5-8435-4d1420cb1b93'
> I0906 04:15:03.31580910 slave.cpp:2118] Queued task 
> 'ct:150467132:0:Job_Task_Test:' for executor 
> 'ct:150467132:0:Job_Task_Test:' of framework 
> 5175f6c9-0617-4145-ab46-3b7e64dc67ea-
> I0906 04:15:03.31623812 docker.cpp:1165] Starting container 
> '7cd8bd78-b20d-4db5-8435-4d1420cb1b93' for task 
> 'ct:150467132:0:Job_Task_Test:' (and executor 
> 'ct:150467132:0:Job_Task_Test:') of framework 
> 5175f6c9-0617-4145-ab46-3b7e64dc67ea-
> I0906 04:15:03.61280710 docker.cpp:803] Checkpointing pid 248 to ' 
> /mesos-data/slave-2/meta/slaves/f20ab78e-acd3-407a-b1b6-47d67a947eff-S1/frameworks/5175f6c9-0617-4145-ab46-3b7e64dc67ea-/executors/ct:150467132:0:Job_Task_Test:/runs/7cd8bd78-b20d-4db5-8435-4d1420cb1b93/pids/forked.pid'
> I0906 04:15:03.64996010 slave.cpp:3385] Got registration for executor 
> 'ct:150467132:0:Job_Task_Test:' of framework 
> 5175f6c9-0617-4145-ab46-3b7e64dc67ea- from executor(1)@20.403.68.700:38740
> I0906 04:15:03.65058411 docker.cpp:1608] Ignoring updating container 
> 7cd8bd78-b20d-4db5-8435-4d1420cb1b93 because resources passed to update are 
> identical to existing resources
> I0906 04:15:03.65070111 slave.cpp:2331] Sending queued task 
> 'ct:150467132:0:Job_Task_Test:' to executor 
> 'ct:150467132:0:Job_Task_Test:' of framework 
> 5175f6c9-0617-4145-ab46-3b7e64dc67ea- at executor(1)@20.403.68.700:38740
> I0906 04:15:05.25510110 slave.cpp:3816] Handling status update 
> TASK_RUNNING (UUID: 35a9a010-d623-45a3-9d1e-bdbc6942129f) for task 
> ct:150467132:0:Job_Task_Test: of framework 
> 5175f6c9-0617-4145-ab46-3b7e64dc67ea- from executor(1)@20.403.68.700:38740
> I0906 04:15:05.25528010 status_update_manager.cpp:323] Received status 
> update TASK_RUNNING (UUID: 35a9a010-d623-45a3-9d1e-bdbc6942129f) for task 
> ct:150467132:0:Job_Task_Test: of framework 
> 5175f6c9-0617-4145-ab46-3b7e64dc67ea-
> I0906 04:15:05.2110 status_update_manager.cpp:832] Checkpointing 
> UPDATE for status update TASK_RUNNING (UUID: 
> 35a9a010-d623-45a3-9d1e-bdbc6942129f) for task 
> ct:150467132:0:Job_Task_Test: of framework 
> 5175f6c9-0617-4145-ab46-3b7e64dc67ea-
> I0906 04:15:05.255697 9 slave.cpp:4256] Forwarding the update 
> TASK_RUNNING (UUID: 35a9a010-d623-45a3-9d1e-bdbc6942129f) for task 
> ct:150467132:0:Job_Task_Test: of framework 
> 5175f6c9-0617-4145-ab46-3b7e64dc67ea- to master@20.403.68.700:5050
> I0906 04:15:05.255803 9 slave.cpp:4166] Sending acknowledgement for 
> status update TASK_RUNNING (UUID: 35a9a010-d623-45a3-9d1e-bdbc6942129f) for 
> task ct:150467132:0:Job_Task_Test: of framework 
> 5175f6c9-0617-4145-ab46-3b7e64dc67ea- to executor(1)@20.403.68.700:38740
> I0906 04:15:05.26008310 status_update_manager.cpp:395] Received status 
> update acknowledgement (UUID: 35a9a010-d623-45a3-9d1e-bdbc6942129f) for task 
> ct:150467132:0:Job_Task_Test: of framework 
> 

[jira] [Commented] (MESOS-7802) Push-commits.py support script is too lenient when determining reviews to close

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155647#comment-16155647
 ] 

Joseph Wu commented on MESOS-7802:
--

Yes, closing just the last review is reasonable.  It'll be up to the 
committer's due diligence to ensure that the last review is actually the review 
they want to close :)

> Push-commits.py support script is too lenient when determining reviews to 
> close
> ---
>
> Key: MESOS-7802
> URL: https://issues.apache.org/jira/browse/MESOS-7802
> Project: Mesos
>  Issue Type: Bug
>Reporter: Joseph Wu
>Priority: Minor
>  Labels: mesosphere, newbie
>
> The support script {{support/push-commits.py}} can be used by committers to 
> push commits and simultaneously close reviews.  However, it is currently 
> quite easy to trick the script into closing unrelated reviews.
> For example, if you have a commit message like:
> {code}
> Referring to multiple reviews in one commit message.
> 
> Review: https://reviews.apache.org/r/1/
> Review: https://reviews.apache.org/r/2/
> Review: https://reviews.apache.org/r/3/
> Review: https://reviews.apache.org/r/4/
> {code}
> The script will do this:
> {code}
> $ support/push-commits.py --dry-run
> Found reviews ['1', '2', '3', '4']
> Pushing commits to apache
> Closing review 1
> Closing review 2
> Closing review 3
> Closing review 4
> {code}
> It is possible for this to happen non-maliciously, if the contributor's 
> review description merely refers to another review in the same format.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7942) Mesos slave - docker job exits normally but reporting as TASK_FAILED

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-7942:
-
   Priority: Major  (was: Blocker)
Description: 
Mesos version - 1.2.1.

Jobs are being scheduled using Chronos. Docker job is being invoked properly, 
but still getting TASK_FAILED error event it completes with exit status ZERO.

Mesos slave logs :-

{code}
I0906 04:15:03.31192810 slave.cpp:1785] Launching task 
'ct:150467132:0:Job_Task_Test:' for framework 
5175f6c9-0617-4145-ab46-3b7e64dc67ea-
I0906 04:15:03.31458410 paths.cpp:547] Trying to chown ' 
/mesos-data/slave-2/slaves/f20ab78e-acd3-407a-b1b6-47d67a947eff-S1/frameworks/5175f6c9-0617-4145-ab46-3b7e64dc67ea-/executors/ct:150467132:0:Job_Task_Test:/runs/7cd8bd78-b20d-4db5-8435-4d1420cb1b93'
 to user 'root'
I0906 04:15:03.31514010 slave.cpp:6479] Launching executor 
'ct:150467132:0:Job_Task_Test:' of framework 
5175f6c9-0617-4145-ab46-3b7e64dc67ea- with resources cpus(*)(allocated: 
*):0.1; mem(*)(allocated: *):32 in work directory ' 
/mesos-data/slave-2/slaves/f20ab78e-acd3-407a-b1b6-47d67a947eff-S1/frameworks/5175f6c9-0617-4145-ab46-3b7e64dc67ea-/executors/ct:150467132:0:Job_Task_Test:/runs/7cd8bd78-b20d-4db5-8435-4d1420cb1b93'
I0906 04:15:03.31580910 slave.cpp:2118] Queued task 
'ct:150467132:0:Job_Task_Test:' for executor 
'ct:150467132:0:Job_Task_Test:' of framework 
5175f6c9-0617-4145-ab46-3b7e64dc67ea-
I0906 04:15:03.31623812 docker.cpp:1165] Starting container 
'7cd8bd78-b20d-4db5-8435-4d1420cb1b93' for task 
'ct:150467132:0:Job_Task_Test:' (and executor 
'ct:150467132:0:Job_Task_Test:') of framework 
5175f6c9-0617-4145-ab46-3b7e64dc67ea-
I0906 04:15:03.61280710 docker.cpp:803] Checkpointing pid 248 to ' 
/mesos-data/slave-2/meta/slaves/f20ab78e-acd3-407a-b1b6-47d67a947eff-S1/frameworks/5175f6c9-0617-4145-ab46-3b7e64dc67ea-/executors/ct:150467132:0:Job_Task_Test:/runs/7cd8bd78-b20d-4db5-8435-4d1420cb1b93/pids/forked.pid'
I0906 04:15:03.64996010 slave.cpp:3385] Got registration for executor 
'ct:150467132:0:Job_Task_Test:' of framework 
5175f6c9-0617-4145-ab46-3b7e64dc67ea- from executor(1)@20.403.68.700:38740
I0906 04:15:03.65058411 docker.cpp:1608] Ignoring updating container 
7cd8bd78-b20d-4db5-8435-4d1420cb1b93 because resources passed to update are 
identical to existing resources
I0906 04:15:03.65070111 slave.cpp:2331] Sending queued task 
'ct:150467132:0:Job_Task_Test:' to executor 
'ct:150467132:0:Job_Task_Test:' of framework 
5175f6c9-0617-4145-ab46-3b7e64dc67ea- at executor(1)@20.403.68.700:38740
I0906 04:15:05.25510110 slave.cpp:3816] Handling status update TASK_RUNNING 
(UUID: 35a9a010-d623-45a3-9d1e-bdbc6942129f) for task 
ct:150467132:0:Job_Task_Test: of framework 
5175f6c9-0617-4145-ab46-3b7e64dc67ea- from executor(1)@20.403.68.700:38740
I0906 04:15:05.25528010 status_update_manager.cpp:323] Received status 
update TASK_RUNNING (UUID: 35a9a010-d623-45a3-9d1e-bdbc6942129f) for task 
ct:150467132:0:Job_Task_Test: of framework 
5175f6c9-0617-4145-ab46-3b7e64dc67ea-
I0906 04:15:05.2110 status_update_manager.cpp:832] Checkpointing UPDATE 
for status update TASK_RUNNING (UUID: 35a9a010-d623-45a3-9d1e-bdbc6942129f) for 
task ct:150467132:0:Job_Task_Test: of framework 
5175f6c9-0617-4145-ab46-3b7e64dc67ea-
I0906 04:15:05.255697 9 slave.cpp:4256] Forwarding the update TASK_RUNNING 
(UUID: 35a9a010-d623-45a3-9d1e-bdbc6942129f) for task 
ct:150467132:0:Job_Task_Test: of framework 
5175f6c9-0617-4145-ab46-3b7e64dc67ea- to master@20.403.68.700:5050
I0906 04:15:05.255803 9 slave.cpp:4166] Sending acknowledgement for status 
update TASK_RUNNING (UUID: 35a9a010-d623-45a3-9d1e-bdbc6942129f) for task 
ct:150467132:0:Job_Task_Test: of framework 
5175f6c9-0617-4145-ab46-3b7e64dc67ea- to executor(1)@20.403.68.700:38740
I0906 04:15:05.26008310 status_update_manager.cpp:395] Received status 
update acknowledgement (UUID: 35a9a010-d623-45a3-9d1e-bdbc6942129f) for task 
ct:150467132:0:Job_Task_Test: of framework 
5175f6c9-0617-4145-ab46-3b7e64dc67ea-
I0906 04:15:05.26011410 status_update_manager.cpp:832] Checkpointing ACK 
for status update TASK_RUNNING (UUID: 35a9a010-d623-45a3-9d1e-bdbc6942129f) for 
task ct:150467132:0:Job_Task_Test: of framework 
5175f6c9-0617-4145-ab46-3b7e64dc67ea-
I0906 04:15:13.09036813 slave.cpp:3816] Handling status update TASK_FAILED 
(UUID: 62ecb989-f260-42b0-ba99-d22fa7210a4c) for task 
ct:150467132:0:Job_Task_Test: of framework 
5175f6c9-0617-4145-ab46-3b7e64dc67ea- from executor(1)@20.403.68.700:38740
I0906 04:15:13.16409613 status_update_manager.cpp:323] Received status 
update TASK_FAILED (UUID: 62ecb989-f260-42b0-ba99-d22fa7210a4c) for task 
ct:150467132:0:Job_Task_Test: of framework 

[jira] [Created] (MESOS-7938) CMake build should give Libevent a hint to use HomeBrew OpenSSL on OSX

[Joseph Wu (JIRA)]

Joseph Wu created MESOS-7938:


 Summary: CMake build should give Libevent a hint to use HomeBrew 
OpenSSL on OSX
 Key: MESOS-7938
 URL: https://issues.apache.org/jira/browse/MESOS-7938
 Project: Mesos
  Issue Type: Improvement
  Components: cmake
Reporter: Joseph Wu


The CMake build currently uses standard CMake modules (i.e. {{find_package}}) 
to find OpenSSL and Libevent installed on the system.  On OSX however, CMake 
will generally find the system installed version of OpenSSL rather than the 
more up-to-date HomeBrew package.

We should consider giving this hint to the Libevent package finder:
{code}
-DOPENSSL_ROOT_DIR=`brew --prefix openssl`
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7935) Build failed: "mesos::uri::fetcher" has not been declared with in-source builds

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-7935:
-
Component/s: (was: build)
 cmake

> Build failed: "mesos::uri::fetcher" has not been declared with in-source 
> builds
> ---
>
> Key: MESOS-7935
> URL: https://issues.apache.org/jira/browse/MESOS-7935
> Project: Mesos
>  Issue Type: Bug
>  Components: cmake
> Environment: macOS 10.12
> GNU/Linux Debian Stretch
>Reporter: Damien Gerard
>  Labels: build
>
> With the latest master the build is broken:
> {noformat}
> cd /Users/damien.gerard/projects/acp/mesos/src && 
> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
>   -DBUILD_FLAGS=\"\" -DBUILD_JAVA_JVM_LIBRARY=\"\" -DHAS_AUTHENTICATION=1 
> -DLIBDIR=\"/usr/local/libmesos\" -DPICOJSON_USE_INT64 
> -DPKGDATADIR=\"/usr/local/share/mesos\" 
> -DPKGLIBEXECDIR=\"/usr/local/libexec/mesos\" -DUSE_CMAKE_BUILD_CONFIG 
> -DUSE_STATIC_LIB -DVERSION=\"1.4.0\" -D__STDC_FORMAT_MACROS 
> -Dmesos_1_4_0_EXPORTS -I/Users/damien.gerard/projects/acp/mesos/include 
> -I/Users/damien.gerard/projects/acp/mesos/include/mesos 
> -I/Users/damien.gerard/projects/acp/mesos/src -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/protobuf-3.3.0/src/protobuf-3.3.0-lib/lib/include
>  -isystem /Users/damien.gerard/projects/acp/mesos/3rdparty/libprocess/include 
> -isystem /usr/local/opt/apr/libexec/include/apr-1 -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/boost-1.53.0/src/boost-1.53.0
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/elfio-3.2/src/elfio-3.2 
> -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/glog-0.3.3/src/glog-0.3.3-lib/lib/include
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/nvml-352.79/src/nvml-352.79 
> -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/picojson-1.3.0/src/picojson-1.3.0
>  -isystem /usr/local/include/subversion-1 -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/stout/include -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/http_parser-2.6.2/src/http_parser-2.6.2
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/concurrentqueue-1.0.0-beta/src/concurrentqueue-1.0.0-beta
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/libev-4.22/src/libev-4.22 
> -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/zookeeper-3.4.8/src/zookeeper-3.4.8/src/c/include
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/zookeeper-3.4.8/src/zookeeper-3.4.8/src/c/generated
>  -isystem 
> /Users/damien.gerard/projects/acp/mesos/3rdparty/leveldb-1.19/src/leveldb-1.19/include
>   -std=c++11 -fPIC   -o 
> CMakeFiles/mesos-1.4.0.dir/slave/containerizer/mesos/provisioner/backends/copy.cpp.o
>  -c 
> /Users/damien.gerard/projects/acp/mesos/src/slave/containerizer/mesos/provisioner/backends/copy.cpp
> /Users/damien.gerard/projects/acp/mesos/src/slave/containerizer/mesos/provisioner/appc/store.cpp:132:46:
>  error: no member named 'fetcher' in namespace 'mesos::uri'; did you mean 
> 'Fetcher'?
>   Try uriFetcher = uri::fetcher::create();
> ~^~~
>  Fetcher
> /Users/damien.gerard/projects/acp/mesos/include/mesos/uri/fetcher.hpp:46:7: 
> note: 'Fetcher' declared here
> class Fetcher
>   ^
> /Users/damien.gerard/projects/acp/mesos/src/slave/containerizer/mesos/provisioner/appc/store.cpp:132:55:
>  error: no member named 'create' in 'mesos::uri::Fetcher'
>   Try uriFetcher = uri::fetcher::create();
> {noformat}
> Both Linux & macOS, not tested elsewhere, on {{master}} and tag 1.4.0-rc3



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7935) CMake build should fail immediately for in-source builds

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-7935:
-
   Shepherd: Joseph Wu
Summary: CMake build should fail immediately for in-source builds  
(was: Build failed: "mesos::uri::fetcher" has not been declared with in-source 
builds)
Description: 
In-source builds are neither recommended or supported.  It is simple enough to 
add a check to fail the build immediately.

---

In-source build of master branch was broken with:

{noformat}
cd /Users/damien.gerard/projects/acp/mesos/src && 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
  -DBUILD_FLAGS=\"\" -DBUILD_JAVA_JVM_LIBRARY=\"\" -DHAS_AUTHENTICATION=1 
-DLIBDIR=\"/usr/local/libmesos\" -DPICOJSON_USE_INT64 
-DPKGDATADIR=\"/usr/local/share/mesos\" 
-DPKGLIBEXECDIR=\"/usr/local/libexec/mesos\" -DUSE_CMAKE_BUILD_CONFIG 
-DUSE_STATIC_LIB -DVERSION=\"1.4.0\" -D__STDC_FORMAT_MACROS 
-Dmesos_1_4_0_EXPORTS -I/Users/damien.gerard/projects/acp/mesos/include 
-I/Users/damien.gerard/projects/acp/mesos/include/mesos 
-I/Users/damien.gerard/projects/acp/mesos/src -isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/protobuf-3.3.0/src/protobuf-3.3.0-lib/lib/include
 -isystem /Users/damien.gerard/projects/acp/mesos/3rdparty/libprocess/include 
-isystem /usr/local/opt/apr/libexec/include/apr-1 -isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/boost-1.53.0/src/boost-1.53.0 
-isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/elfio-3.2/src/elfio-3.2 
-isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/glog-0.3.3/src/glog-0.3.3-lib/lib/include
 -isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/nvml-352.79/src/nvml-352.79 
-isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/picojson-1.3.0/src/picojson-1.3.0
 -isystem /usr/local/include/subversion-1 -isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/stout/include -isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/http_parser-2.6.2/src/http_parser-2.6.2
 -isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/concurrentqueue-1.0.0-beta/src/concurrentqueue-1.0.0-beta
 -isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/libev-4.22/src/libev-4.22 
-isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/zookeeper-3.4.8/src/zookeeper-3.4.8/src/c/include
 -isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/zookeeper-3.4.8/src/zookeeper-3.4.8/src/c/generated
 -isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/leveldb-1.19/src/leveldb-1.19/include
  -std=c++11 -fPIC   -o 
CMakeFiles/mesos-1.4.0.dir/slave/containerizer/mesos/provisioner/backends/copy.cpp.o
 -c 
/Users/damien.gerard/projects/acp/mesos/src/slave/containerizer/mesos/provisioner/backends/copy.cpp
/Users/damien.gerard/projects/acp/mesos/src/slave/containerizer/mesos/provisioner/appc/store.cpp:132:46:
 error: no member named 'fetcher' in namespace 'mesos::uri'; did you mean 
'Fetcher'?
  Try uriFetcher = uri::fetcher::create();
~^~~
 Fetcher
/Users/damien.gerard/projects/acp/mesos/include/mesos/uri/fetcher.hpp:46:7: 
note: 'Fetcher' declared here
class Fetcher
  ^
/Users/damien.gerard/projects/acp/mesos/src/slave/containerizer/mesos/provisioner/appc/store.cpp:132:55:
 error: no member named 'create' in 'mesos::uri::Fetcher'
  Try uriFetcher = uri::fetcher::create();
{noformat}

Both Linux & macOS, not tested elsewhere, on {{master}} and tag 1.4.0-rc3

  was:
With the latest master the build is broken:

{noformat}
cd /Users/damien.gerard/projects/acp/mesos/src && 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
  -DBUILD_FLAGS=\"\" -DBUILD_JAVA_JVM_LIBRARY=\"\" -DHAS_AUTHENTICATION=1 
-DLIBDIR=\"/usr/local/libmesos\" -DPICOJSON_USE_INT64 
-DPKGDATADIR=\"/usr/local/share/mesos\" 
-DPKGLIBEXECDIR=\"/usr/local/libexec/mesos\" -DUSE_CMAKE_BUILD_CONFIG 
-DUSE_STATIC_LIB -DVERSION=\"1.4.0\" -D__STDC_FORMAT_MACROS 
-Dmesos_1_4_0_EXPORTS -I/Users/damien.gerard/projects/acp/mesos/include 
-I/Users/damien.gerard/projects/acp/mesos/include/mesos 
-I/Users/damien.gerard/projects/acp/mesos/src -isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/protobuf-3.3.0/src/protobuf-3.3.0-lib/lib/include
 -isystem /Users/damien.gerard/projects/acp/mesos/3rdparty/libprocess/include 
-isystem /usr/local/opt/apr/libexec/include/apr-1 -isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/boost-1.53.0/src/boost-1.53.0 
-isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/elfio-3.2/src/elfio-3.2 
-isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/glog-0.3.3/src/glog-0.3.3-lib/lib/include
 -isystem 
/Users/damien.gerard/projects/acp/mesos/3rdparty/nvml-352.79/src/nvml-352.79 
-isystem 

[jira] [Commented] (MESOS-6721) Group source files into folders for IDE's

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151800#comment-16151800
 ] 

Joseph Wu commented on MESOS-6721:
--

Reviews that removed the non-working existing source grouping logic:
{code}
commit b8d187f338efb27d11f85921955dc0ee93ae501a
Author: Andrew Schwartzmeyer 
Date:   Sat Sep 2 09:34:32 2017 -0700

CMake: Removed `GroupSource.cmake`.

While we want to group sources, this current solution is not
working as intended.  We will consider looking into grouping
source again in future.

Review: https://reviews.apache.org/r/61351/

commit 96866d2c6105cda479a33371401377a77281a0e9
Author: Andrew Schwartzmeyer 
Date:   Sat Sep 2 09:07:34 2017 -0700

CMake: Removed `GroupSource` usage from `libprocess`.

Source grouping is a cosmetic change that only has an effect if
you load this project into an IDE (such as XCode or Visual Studio).

The existing source grouping is brittle and is not vital for the
CMake build system MVP, so we are removing it to reduce code churn.

Review: https://reviews.apache.org/r/61350/

commit ae778e46ecdb8096ae8d69e584a69114d2226fd0
Author: Andrew Schwartzmeyer 
Date:   Mon Aug 7 18:09:48 2017 -0700

CMake: Removed `GroupSource` from `stout`.

Source grouping is a cosmetic change that only has an effect if
you load this project into an IDE (such as XCode or Visual Studio).

The existing source grouping is brittle and is not vital for the
CMake build system MVP, so we are removing it to reduce code churn.

Review: https://reviews.apache.org/r/61287/
{code}

> Group source files into folders for IDE's
> -
>
> Key: MESOS-6721
> URL: https://issues.apache.org/jira/browse/MESOS-6721
> Project: Mesos
>  Issue Type: Bug
>  Components: cmake
>Reporter: Alex Clemmer
>Assignee: Alex Clemmer
>  Labels: cmake, microsoft
>
> CMake has good facilities for organizing source files in a project into 
> folders, but we don't really make use of them. This is especially bad for 
> IDEs like XCode and Visual Studio, where the source files will just end up in 
> a folder with literally everything that's included.
> For every executable and library we make, we should do something like this 
> (and it might be wrong, because my memory is hazy here):
> ```
> set_property(TARGET ${AGENT_TARGET} PROPERTY FOLDER "src/slave")
> ```



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-7929) `Metrics()` hangs on second call on Windows

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu reassigned MESOS-7929:


Assignee: Joseph Wu

> `Metrics()` hangs on second call on Windows
> ---
>
> Key: MESOS-7929
> URL: https://issues.apache.org/jira/browse/MESOS-7929
> Project: Mesos
>  Issue Type: Bug
> Environment: Windows 10 Enterprise Build 15063 (and also confirmed on 
> 14393).
>Reporter: Andrew Schwartzmeyer
>Assignee: Joseph Wu
>Priority: Critical
>  Labels: windows
>
> An unfortunately difficult to debug problem has cropped up on Windows. While 
> running the {{mesos-tests}} they will hang at:
> {noformat}
> [==] Running 2 tests from 2 test cases.
> [--] Global test environment set-up.
> [--] 1 test from FetcherTest
> [ RUN  ] FetcherTest.MalformedURI
> [   OK ] FetcherTest.MalformedURI (48 ms)
> [--] 1 test from FetcherTest (63 ms total)
> [--] 1 test from GarbageCollectorTest
> [ RUN  ] GarbageCollectorTest.Schedule
> C:\Users\andschwa\src\mesos-master\src\tests\utils.cpp(64): error: Failed to 
> wait 15secs for response
> C:\Users\andschwa\src\mesos-master\src\tests\utils.cpp(65): error: Failed to 
> wait 15secs for response
> {noformat}
> {{GarbageCollectorTest.Schedule}} is the first test that will hang in an 
> unfiltered run of mesos-tests.
> This can be minimally reproduced by running any two tests which call 
> {{Metrics()}} from {{utils.cpp}}. The following have been confirmed:
> {noformat}
> --gtest_filter="GarbageCollectorTest.Schedule:HierarchicalAllocatorTest.OfferFilter"
> --gtest_filter="GarbageCollectorTest.Schedule:FetcherTest.MalformedURI"
> --gtest_filter="HierarchicalAllocatorTest.OfferFilter:FetcherTest.MalformedURI"
> {noformat}
> The second test will hang (indicating a race condition), waiting for a 
> {{GET}} to {{/metrics/snapshot}} that never returns.
> There appears to be a timing problem to this bug as well. If your CPU is 
> heavily utilized (say, by running another build in the background), the tests 
> will pass. They will pass if you attach Application Verifier to 
> {{mesos-tests.exe}}, which slows down execution enough. Very slow machines 
> (such as those used for CI) will also not exhibit this hang. Increasing the 
> log verbosity using {{GLOG_v=2}} or higher will make it disappear (although 
> {{--verbose}} has no effect).
> Oddly, the bug will reproduce under the Visual Studio debugger, but all it 
> shows us is a pending future waiting for the metrics request to come back.
> In {{metrics.cpp}} there is a note that the request might timeout, but we're 
> unsure if this is the same problem, or a different problem manifesting in the 
> same way:
> {noformat}
>   // TODO(neilc): This request might timeout if the current value of a
>   // metric cannot be determined. In tests, a common cause for this is
>   // MESOS-6231 when multiple scheduler drivers are in use.
> {noformat}
> A {{git bisect}} revealed that:
> {noformat}
> 20c5311434e45a631ffc6036d327e00b2228ad26 is the first bad commit
> commit 20c5311434e45a631ffc6036d327e00b2228ad26
> Author: James Peach 
> Date:   Tue Aug 22 16:19:47 2017 -0700
> Added agent garbage collection metrics.
> Added some basic sandbox garbage collection metrics to track the number
> of successful, failed and pending path removals.
> Review: https://reviews.apache.org/r/61260/
> {noformat}
> Caused this bug to appear (but does not necessarily mean it created the bug). 
> Reverting this commit allows all the tests to pass, but we believe this just 
> hides the bug.
> This bug has reproduced on Windows machines with and without Docker (and 
> Windows containers) installed. (I only mention this because it was a variable 
> on my machine when the bug first appeared, but have since ruled it out as 
> relevant.)
> We do not think that it is specific to {{libevent}}, as the bug does not 
> appear to reproduce on a Linux VM built with {{libevent}} instead of 
> {{libev}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7901) Logrotation not working

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133796#comment-16133796
 ] 

Joseph Wu commented on MESOS-7901:
--

You need to configure the logrotate module with some additional options.  
Specifically:
{code}
"logrotate_stdout_options": "rotate 9"
"logrotate_stderr_options": "rotate 9"
{code}

This sets the default logrotation to 10 files of 10 MB each.  If you don't 
include any rotation rules, using the {{logrotate}} binary is a noop.

> Logrotation not working 
> 
>
> Key: MESOS-7901
> URL: https://issues.apache.org/jira/browse/MESOS-7901
> Project: Mesos
>  Issue Type: Bug
>  Components: agent
>Affects Versions: 1.0.1
>Reporter: BhardRahul
>  Labels: logrotate, mesos
> Attachments: screenshot-1.png
>
>
> Hi,
> Here are our env details:
> *Mesos -* 1.0.1
> *System Logrotate -*  3.8.6-12.el7.x86_64
> Issue:
> We have configured logrotation following these documents:
> http://mesos.apache.org/documentation/latest/logging/
> http://continuousfailure.com/post/mesos27_logging/
> The configuration looks successful. We can see
>   - Mesos service started fine with logrotate module:
> {noformat}
> ]$ ps -ef | grep mesos-slave
> root  41061  1  1 09:10 ?00:00:26 /usr/sbin/mesos-slave 
> --containerizers=docker,mesos 
> --container_logger=org_apache_mesos_LogrotateContainerLogger 
> --executor_registration_timeout=2mins --hostname=xx.xxx.xx.xx 
> --ip=xx.xx.xx.xx --modules=file:///etc/mesos-slave-logrotate.json 
> --work_dir=/xx/x-x --attributes=rack:;zone: 
> --resources=ports:[10-65535]
> {noformat}
>   - Mesos-logrotate process for stdout & stderr files
> {noformat}
> ]$  ps -ef | grep mesos | grep logrotate
> root   8398  1  0 Aug17 ?00:00:00 mesos-logrotate-logger 
> --help=false 
> --log_filename=/xxx/xx/slaves/2dabe42d-afbd-4c5f-8bfc-d0233d366d02-S160/frameworks/cad4ec2a-4c09-45b2-8170-d0d62a2d06f5-/executors/xxx_xxx_xxx-x-v2.2c05d324-83c3-11e7-b256-06af9b31c5f7/runs/9a32d4cd-72f5-4494-93e3-14adf9d5620d/stdout
>  --logrotate_path=logrotate --max_size=10MB
> root   8399  1  0 Aug17 ?00:00:00 mesos-logrotate-logger 
> --help=false 
> --log_filename=/xxx/xxx/slaves/2dabe42d-afbd-4c5f-8bfc-d0233d366d02-S160/frameworks/cad4ec2a-4c09-45b2-8170-d0d62a2d06f5-/executors/xxx_xxx_xxx-x-v2.2c05d324-83c3-11e7-b256-06af9b31c5f7/runs/9a32d4cd-72f5-4494-93e3-14adf9d5620d/stderr
>  --logrotate_path=logrotate --max_size=10MB
> {noformat} 
> But logrotation is not happening when the size of 'stdout' crossed 10MB.Its 
> size is 1.4GB still no rotation.
> {noformat}
> ]$ du -sh 
> /xxx/-/slaves/2dabe42d-afbd-4c5f-8bfc-d0233d366d02-S160/frameworks/cad4ec2a-4c09-45b2-8170-d0d62a2d06f5-/executors/xxx_xxx_xxx-x-v2.2c05d324-83c3-11e7-b256-06af9b31c5f7/runs/9a32d4cd-72f5-4494-93e3-14adf9d5620d/stdout
> 1.4G 
> /xxx/-/slaves/2dabe42d-afbd-4c5f-8bfc-d0233d366d02-S160/frameworks/cad4ec2a-4c09-45b2-8170-d0d62a2d06f5-/executors/xxx_xxx_xxx-x-v2.2c05d324-83c3-11e7-b256-06af9b31c5f7/runs/9a32d4cd-72f5-4494-93e3-14adf9d5620d/stdout
> {noformat}
> Please assist.
> Thanks
> Rahul



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7894) Mesos UI - Disk:Used Field isn't populated with Docker Container Runtime.

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128868#comment-16128868
 ] 

Joseph Wu commented on MESOS-7894:
--

This is due to how the disk resource field is populated on the agent's resource 
statistics endpoint.

The MesosContainerizer (UCR) will fetch usage metrics from each isolator (which 
includes a {{disk/du}} isolator reporting disk usage) and then merge these 
results together, giving you the 128 MB you expect.

The DockerContainerizer just reports some cgroups stats, which does not include 
disk info.

> Mesos UI - Disk:Used Field isn't populated with Docker Container Runtime.
> -
>
> Key: MESOS-7894
> URL: https://issues.apache.org/jira/browse/MESOS-7894
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 1.2.2
> Environment: DC/OS 1.9.2 (CentOS 7.3, Docker 1.13.1, Mesos 1.2.2, 
> Marathon 1.4.5)
>Reporter: Justin Lee
>Priority: Minor
> Attachments: UCR on left, Docker on right.png
>
>
> If you use the Docker container runtime, the 'Disk' 'Used' field never gets 
> populated in the Mesos UI (on the executor/task page).
> Steps to Reproduce:
> in DC/OS 1.9.2, deploy two apps:
> {code:javascript}
> {
>   "id": "/dummy-disk-docker",
>   "cmd": "dd if=/dev/zero of=$MESOS_SANDBOX/testfile bs=128M count=1; tail -f 
> /dev/null",
>   "instances": 1,
>   "cpus": 0.1,
>   "mem": 256,
>   "disk": 150,
>   "container": {
> "type": "DOCKER",
> "docker": {
>   "image": "alpine"
> }
>   }
> }
> {code}
> {code:javascript}
> {
>   "id": "/dummy-disk-ucr",
>   "cmd": "dd if=/dev/zero of=$MESOS_SANDBOX/testfile bs=128M count=1; tail -f 
> /dev/null",
>   "instances": 1,
>   "cpus": 0.1,
>   "mem": 256,
>   "disk": 150,
>   "container": {
> "type": "MESOS"
> "docker": {
>   "image": "alpine"
> }
>   }
> }
> {code}
> Wait for them the deploy.  
> Then, navigate to the mesos UI, and go to the executor/task page for the two 
> tasks.
> On the UCR task, eventually the "Used Disk" field should populate with 128 MB 
> (the size of the dummy file).
> The same field on the Docker task will never get populated.
> Both containers are writing to the same location on the agent filesystem 
> ({{/var/lib/mesos/slave/slaves//frameworks//executors//runs/latest}}),
>  but only one reports the data through the UI.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7886) Add master hook for setting environment variables

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126195#comment-16126195
 ] 

Joseph Wu commented on MESOS-7886:
--

Security-wise, logging isn't the only concern.  But it is the most talked-about 
concern (because it is easy to spot).  Environment variables can usually be 
inspected by anyone that has access to the box, including other tasks on the 
box.  If you run docker tasks, environment variables can be seen in the {{ps 
aux}} output, as they are passed in via the command line (MESOS-6951).

Business-logic-wise, some of the concern is with maintaining or updating the 
hook itself.  It sounds like less work to update a set of three/five masters 
when you update the hook, but updating masters is actually more disruptive than 
a rolling restart of hundreds/thousands of agents :)


> Add master hook for setting environment variables
> -
>
> Key: MESOS-7886
> URL: https://issues.apache.org/jira/browse/MESOS-7886
> Project: Mesos
>  Issue Type: Improvement
>  Components: modules
>Reporter: Matthew Mead-Briggs
>
> At Yelp we're planning to integrate our secret store with our platform as a 
> service which runs on Mesos.
> I was hoping to write a module to "inject" environment variables on the 
> master side but the necessary hook doesn't currently exist. Such a hook 
> already exists on the slave side. However, for this integration that would 
> require me to give all the agents access to the secret store and I'd much 
> prefer to limit this to the master side.
> There is already a hook for adding labels:
> https://github.com/apache/mesos/blob/72752fc6deb8ebcbfbd5448dc599ef3774339d31/include/mesos/hook.hpp#L44-L48
> So it seems it should be pretty easy to add one for setting environment 
> variables too? I had a crack the other day but although I got my code to 
> compile something was not working at runtime (note: I'm not a C++ dev). Is 
> there any reason why we wouldn't want such a hook? If anyone can confirm that 
> it's a sane thing to add then I'd be happy to spend some time trying to get 
> it working (although I may need some help)!



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7886) Add master hook for setting environment variables

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126050#comment-16126050
 ] 

Joseph Wu commented on MESOS-7886:
--

>From a security perspective, putting secrets in environment variables is not 
>ideal (but it is admittedly pretty common).  There are a few places in the 
>Mesos code (in older versions) where environment variables are printed to logs 
>or stderr.

>From a historical perspective, the master generally limits itself to 
>coordinating frameworks and agents, but stays out of the business logic needed 
>to run tasks.  This is mostly because heterogeneous clusters can have many 
>different agent configurations; and having the master keep track of how to 
>handle each configuration may become onerous.

> Add master hook for setting environment variables
> -
>
> Key: MESOS-7886
> URL: https://issues.apache.org/jira/browse/MESOS-7886
> Project: Mesos
>  Issue Type: Improvement
>  Components: modules
>Reporter: Matthew Mead-Briggs
>
> At Yelp we're planning to integrate our secret store with our platform as a 
> service which runs on Mesos.
> I was hoping to write a module to "inject" environment variables on the 
> master side but the necessary hook doesn't currently exist. Such a hook 
> already exists on the slave side. However, for this integration that would 
> require me to give all the agents access to the secret store and I'd much 
> prefer to limit this to the master side.
> There is already a hook for adding labels:
> https://github.com/apache/mesos/blob/72752fc6deb8ebcbfbd5448dc599ef3774339d31/include/mesos/hook.hpp#L44-L48
> So it seems it should be pretty easy to add one for setting environment 
> variables too? I had a crack the other day but although I got my code to 
> compile something was not working at runtime (note: I'm not a C++ dev). Is 
> there any reason why we wouldn't want such a hook? If anyone can confirm that 
> it's a sane thing to add then I'd be happy to spend some time trying to get 
> it working (although I may need some help)!



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-3576) Audit CMake linking flags

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-3576:
-
Shepherd: Joseph Wu  (was: Joris Van Remoortere)

> Audit CMake linking flags
> -
>
> Key: MESOS-3576
> URL: https://issues.apache.org/jira/browse/MESOS-3576
> Project: Mesos
>  Issue Type: Bug
>  Components: cmake
>Reporter: Alex Clemmer
>Assignee: Andrew Schwartzmeyer
>  Labels: build, mesosphere
>
> If you look at the linking flags for autoconf's stout tests build:
> ```
> ./.libs/libgmock.a glog-0.3.3/.libs/libglog.a -lgflags 
> protobuf-2.5.0/src/.libs/libprotobuf.a -lpthread -ldl -lz 
> /usr/lib/x86_64-linux-gnu/libcurl-nss.so 
> /usr/lib/x86_64-linux-gnu/libsvn_delta-1.so 
> /usr/lib/x86_64-linux-gnu/libsvn_subr-1.so 
> /usr/lib/x86_64-linux-gnu/libapr-1.so -lrt -pthread
> ```
> you'll notice that they are much more concise than our CMake build:
> ```
> -L/usr/lib/x86_64-linux-gnu/libapr-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_client-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_delta-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_diff-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_fs-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_fs_fs-1.a  
> -L/usr/lib/x86_64-linux-gnu/libsvn_fs_util-1.a  
> -L/usr/lib/x86_64-linux-gnu/libsvn_ra-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_ra_local-1.a  
> -L/usr/lib/x86_64-linux-gnu/libsvn_ra_serf-1.a  
> -L/usr/lib/x86_64-linux-gnu/libsvn_ra_svn-1.a  
> -L/usr/lib/x86_64-linux-gnu/libsvn_repos-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_subr-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_wc-1.so  
> -L/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/gmock-1.7.0/src/gmock-1.7.0-lib/lib
>   
> -L/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/gmock-1.7.0/src/gmock-1.7.0-build/gtest/lib/.libs
>   
> -L/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/glog-0.3.3/src/glog-0.3.3-lib/lib/lib
>   
> -L/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protobuf-2.5.0-lib/lib/lib
>  -rdynamic -lpthread -lgmock -lsvn_client-1 -lsvn_delta-1 -lsvn_diff-1 
> -lsvn_fs-1 -Wl,-Bstatic -lsvn_fs_fs-1 -lsvn_fs_util-1 -Wl,-Bdynamic 
> -lsvn_ra-1 -Wl,-Bstatic -lsvn_ra_local-1 -lsvn_ra_serf-1 -lsvn_ra_svn-1 
> -Wl,-Bdynamic -lsvn_repos-1 -lsvn_subr-1 -lsvn_wc-1 -lglog -lprotobuf -lgtest 
> -ldl -lapr-1 -lrt 
> -Wl,-rpath,/usr/lib/x86_64-linux-gnu/libapr-1.so:/usr/lib/x86_64-linux-gnu/libsvn_client-1.so:/usr/lib/x86_64-linux-gnu/libsvn_delta-1.so:/usr/lib/x86_64-linux-gnu/libsvn_diff-1.so:/usr/lib/x86_64-linux-gnu/libsvn_fs-1.so:/usr/lib/x86_64-linux-gnu/libsvn_fs_fs-1.a:/usr/lib/x86_64-linux-gnu/libsvn_fs_util-1.a:/usr/lib/x86_64-linux-gnu/libsvn_ra-1.so:/usr/lib/x86_64-linux-gnu/libsvn_ra_local-1.a:/usr/lib/x86_64-linux-gnu/libsvn_ra_serf-1.a:/usr/lib/x86_64-linux-gnu/libsvn_ra_svn-1.a:/usr/lib/x86_64-linux-gnu/libsvn_repos-1.so:/usr/lib/x86_64-linux-gnu/libsvn_subr-1.so:/usr/lib/x86_64-linux-gnu/libsvn_wc-1.so:/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/gmock-1.7.0/src/gmock-1.7.0-lib/lib:/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/gmock-1.7.0/src/gmock-1.7.0-build/gtest/lib/.libs:/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/glog-0.3.3/src/glog-0.3.3-lib/lib/lib:/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protobuf-2.5.0-lib/lib/lib
> ```
> We need to (1) audit this so that we are confident the linking process works 
> like we want it to, and (2) make sure we don't triple link dependencies.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-6390) Ensure Python support scripts are linted

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-6390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121894#comment-16121894
 ] 

Joseph Wu commented on MESOS-6390:
--

{code}
commit d04ab2096169513561d20a414c67ed1aaed0ecd7
Author: Armand Grillet 
Date:   Thu Aug 10 09:38:43 2017 -0700

Linted support/test-upgrade.py.

This will allow us to use PyLint on the
entire support directory in the future.

Review: https://reviews.apache.org/r/60235/
{code}

> Ensure Python support scripts are linted
> 
>
> Key: MESOS-6390
> URL: https://issues.apache.org/jira/browse/MESOS-6390
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Benjamin Bannier
>Assignee: Armand Grillet
>  Labels: newbie, python
>
> Currently {{support/mesos-style.py}} does not lint files under {{support/}}. 
> This is mostly due to the fact that these scripts are too inconsistent 
> style-wise that they wouldn't even pass the linter now.
> We should clean up all Python scripts under {{support/}} so they pass the 
> Python linter, and activate that directory in the linter for future 
> additions. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-6350) Raise minimum required cmake version

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-6350:
-
Labels: mesosphere microsoft tech-debt  (was: cmake mesosphere microsoft 
tech-debt)

> Raise minimum required cmake version
> 
>
> Key: MESOS-6350
> URL: https://issues.apache.org/jira/browse/MESOS-6350
> Project: Mesos
>  Issue Type: Improvement
>  Components: cmake
>Reporter: Benjamin Bannier
>Assignee: Andrew Schwartzmeyer
>  Labels: mesosphere, microsoft, tech-debt
>
> We currently require at least cmake-2.8 which had its first point release 
> 2010 and last update 2013. Meanwhile upstream is preparing the release of 
> 3.7.0. While cmake support in Mesos is still experimental we should evaluate 
> how much we can increase the minimal required version so we are not locked 
> into an old version lacking desirable features.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-3576) Audit CMake linking flags

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-3576:
-
 Labels: build mesosphere  (was: build cmake mesosphere)
Component/s: (was: build)
 cmake

> Audit CMake linking flags
> -
>
> Key: MESOS-3576
> URL: https://issues.apache.org/jira/browse/MESOS-3576
> Project: Mesos
>  Issue Type: Bug
>  Components: cmake
>Reporter: Alex Clemmer
>Assignee: Andrew Schwartzmeyer
>  Labels: build, mesosphere
>
> If you look at the linking flags for autoconf's stout tests build:
> ```
> ./.libs/libgmock.a glog-0.3.3/.libs/libglog.a -lgflags 
> protobuf-2.5.0/src/.libs/libprotobuf.a -lpthread -ldl -lz 
> /usr/lib/x86_64-linux-gnu/libcurl-nss.so 
> /usr/lib/x86_64-linux-gnu/libsvn_delta-1.so 
> /usr/lib/x86_64-linux-gnu/libsvn_subr-1.so 
> /usr/lib/x86_64-linux-gnu/libapr-1.so -lrt -pthread
> ```
> you'll notice that they are much more concise than our CMake build:
> ```
> -L/usr/lib/x86_64-linux-gnu/libapr-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_client-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_delta-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_diff-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_fs-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_fs_fs-1.a  
> -L/usr/lib/x86_64-linux-gnu/libsvn_fs_util-1.a  
> -L/usr/lib/x86_64-linux-gnu/libsvn_ra-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_ra_local-1.a  
> -L/usr/lib/x86_64-linux-gnu/libsvn_ra_serf-1.a  
> -L/usr/lib/x86_64-linux-gnu/libsvn_ra_svn-1.a  
> -L/usr/lib/x86_64-linux-gnu/libsvn_repos-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_subr-1.so  
> -L/usr/lib/x86_64-linux-gnu/libsvn_wc-1.so  
> -L/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/gmock-1.7.0/src/gmock-1.7.0-lib/lib
>   
> -L/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/gmock-1.7.0/src/gmock-1.7.0-build/gtest/lib/.libs
>   
> -L/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/glog-0.3.3/src/glog-0.3.3-lib/lib/lib
>   
> -L/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protobuf-2.5.0-lib/lib/lib
>  -rdynamic -lpthread -lgmock -lsvn_client-1 -lsvn_delta-1 -lsvn_diff-1 
> -lsvn_fs-1 -Wl,-Bstatic -lsvn_fs_fs-1 -lsvn_fs_util-1 -Wl,-Bdynamic 
> -lsvn_ra-1 -Wl,-Bstatic -lsvn_ra_local-1 -lsvn_ra_serf-1 -lsvn_ra_svn-1 
> -Wl,-Bdynamic -lsvn_repos-1 -lsvn_subr-1 -lsvn_wc-1 -lglog -lprotobuf -lgtest 
> -ldl -lapr-1 -lrt 
> -Wl,-rpath,/usr/lib/x86_64-linux-gnu/libapr-1.so:/usr/lib/x86_64-linux-gnu/libsvn_client-1.so:/usr/lib/x86_64-linux-gnu/libsvn_delta-1.so:/usr/lib/x86_64-linux-gnu/libsvn_diff-1.so:/usr/lib/x86_64-linux-gnu/libsvn_fs-1.so:/usr/lib/x86_64-linux-gnu/libsvn_fs_fs-1.a:/usr/lib/x86_64-linux-gnu/libsvn_fs_util-1.a:/usr/lib/x86_64-linux-gnu/libsvn_ra-1.so:/usr/lib/x86_64-linux-gnu/libsvn_ra_local-1.a:/usr/lib/x86_64-linux-gnu/libsvn_ra_serf-1.a:/usr/lib/x86_64-linux-gnu/libsvn_ra_svn-1.a:/usr/lib/x86_64-linux-gnu/libsvn_repos-1.so:/usr/lib/x86_64-linux-gnu/libsvn_subr-1.so:/usr/lib/x86_64-linux-gnu/libsvn_wc-1.so:/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/gmock-1.7.0/src/gmock-1.7.0-lib/lib:/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/gmock-1.7.0/src/gmock-1.7.0-build/gtest/lib/.libs:/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/glog-0.3.3/src/glog-0.3.3-lib/lib/lib:/home/joris/projects/mesos/build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protobuf-2.5.0-lib/lib/lib
> ```
> We need to (1) audit this so that we are confident the linking process works 
> like we want it to, and (2) make sure we don't triple link dependencies.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-6350) Raise minimum required cmake version

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-6350:
-
Component/s: (was: build)
 cmake

> Raise minimum required cmake version
> 
>
> Key: MESOS-6350
> URL: https://issues.apache.org/jira/browse/MESOS-6350
> Project: Mesos
>  Issue Type: Improvement
>  Components: cmake
>Reporter: Benjamin Bannier
>Assignee: Andrew Schwartzmeyer
>  Labels: cmake, mesosphere, microsoft, tech-debt
>
> We currently require at least cmake-2.8 which had its first point release 
> 2010 and last update 2013. Meanwhile upstream is preparing the release of 
> 3.7.0. While cmake support in Mesos is still experimental we should evaluate 
> how much we can increase the minimal required version so we are not locked 
> into an old version lacking desirable features.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-7834) CMake does not set default --launcher_dir correctly

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu reassigned MESOS-7834:


Shepherd: Joseph Wu
Assignee: Andrew Schwartzmeyer
  Labels: cmake  (was: cmake windows)

> CMake does not set default --launcher_dir correctly
> ---
>
> Key: MESOS-7834
> URL: https://issues.apache.org/jira/browse/MESOS-7834
> Project: Mesos
>  Issue Type: Bug
>  Components: agent, cmake
> Environment: Windows 10
>Reporter: Andrew Schwartzmeyer
>Assignee: Andrew Schwartzmeyer
>  Labels: cmake
>
> If the mesos-agent is started without {{--launcher_dir}} and was built using 
> CMake, the default location used to find the other executables (e.g. 
> mesos-containerizer and mesos-executor) does not exist, because it is 
> {{WARNINGDONOTUSEME/mesos-containerizer}}.
> This {{WARNINGDONOTUSEME}} is being set in several variables in CMake:
> {noformat}
> cmake/CompilationConfigure.cmake
> 313:  set(EXEC_INSTALL_PREFIX "WARNINGDONOTUSEME")
> 314:  set(LIBEXEC_INSTALL_DIR "WARNINGDONOTUSEME")
> 315:  set(PKG_LIBEXEC_INSTALL_DIR "WARNINGDONOTUSEME")
> 316:  set(LIB_INSTALL_DIR "WARNINGDONOTUSEME")
> 317:  set(TEST_LIB_EXEC_DIR   "WARNINGDONOTUSEME")
> 318:  set(PKG_MODULE_DIR  "WARNINGDONOTUSEME")
> 319:  set(S_BIN_DIR   "WARNINGDONOTUSEME")
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-6721) Group source files into folders for IDE's

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-6721:
-
Summary: Group source files into folders for IDE's  (was: Cause source 
files to be correctly grouped into folders)

> Group source files into folders for IDE's
> -
>
> Key: MESOS-6721
> URL: https://issues.apache.org/jira/browse/MESOS-6721
> Project: Mesos
>  Issue Type: Bug
>  Components: cmake
>Reporter: Alex Clemmer
>Assignee: Alex Clemmer
>  Labels: cmake, microsoft
>
> CMake has good facilities for organizing source files in a project into 
> folders, but we don't really make use of them. This is especially bad for 
> IDEs like XCode and Visual Studio, where the source files will just end up in 
> a folder with literally everything that's included.
> For every executable and library we make, we should do something like this 
> (and it might be wrong, because my memory is hazy here):
> ```
> set_property(TARGET ${AGENT_TARGET} PROPERTY FOLDER "src/slave")
> ```



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-6671) External 3rdparty deps are not built with the configured compiler in cmake build

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu reassigned MESOS-6671:


Shepherd: Joseph Wu
Assignee: Andrew Schwartzmeyer

> External 3rdparty deps are not built with the configured compiler in cmake 
> build
> 
>
> Key: MESOS-6671
> URL: https://issues.apache.org/jira/browse/MESOS-6671
> Project: Mesos
>  Issue Type: Bug
>  Components: cmake
>Reporter: Benjamin Bannier
>Assignee: Andrew Schwartzmeyer
>  Labels: cmake
>
> CMake permits users to modify the compiler for e.g., C++ source files by 
> setting {{CMAKE_CXX_COMPILER}}. Additionally, to use compiler wrappers like 
> ccache or distcc modern cmake versions allow to specify the wrapper by 
> setting e.g., {{CMAKE_CXX_COMPILER_LAUNCHER}}.
> The current Mesos cmake ignores both these variables when building external 
> 3rdparty autotools-based dependencies, and the only way to overwrite the 
> compiler there would be to set e.g., {{CXX='ccache clang++'}} and {{make}} 
> time. This is undesirable as it is too easy to introduce inconsistent 
> compiler settings.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-6350) Raise minimum required cmake version

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-6350:
-
Shepherd: Joseph Wu

> Raise minimum required cmake version
> 
>
> Key: MESOS-6350
> URL: https://issues.apache.org/jira/browse/MESOS-6350
> Project: Mesos
>  Issue Type: Improvement
>  Components: build
>Reporter: Benjamin Bannier
>Assignee: Andrew Schwartzmeyer
>  Labels: cmake, mesosphere, microsoft, tech-debt
>
> We currently require at least cmake-2.8 which had its first point release 
> 2010 and last update 2013. Meanwhile upstream is preparing the release of 
> 3.7.0. While cmake support in Mesos is still experimental we should evaluate 
> how much we can increase the minimal required version so we are not locked 
> into an old version lacking desirable features.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-6350) Raise minimum required cmake version

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-6350:
-
 Labels: cmake mesosphere microsoft tech-debt  (was: mesosphere 
microsoft tech-debt)
Component/s: (was: cmake)
 build

> Raise minimum required cmake version
> 
>
> Key: MESOS-6350
> URL: https://issues.apache.org/jira/browse/MESOS-6350
> Project: Mesos
>  Issue Type: Improvement
>  Components: build
>Reporter: Benjamin Bannier
>Assignee: Andrew Schwartzmeyer
>  Labels: cmake, mesosphere, microsoft, tech-debt
>
> We currently require at least cmake-2.8 which had its first point release 
> 2010 and last update 2013. Meanwhile upstream is preparing the release of 
> 3.7.0. While cmake support in Mesos is still experimental we should evaluate 
> how much we can increase the minimal required version so we are not locked 
> into an old version lacking desirable features.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-5656) Incomplete modelling of 3rdparty dependencies in cmake build

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-5656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-5656:
-
Shepherd: Joseph Wu

> Incomplete modelling of 3rdparty dependencies in cmake build
> 
>
> Key: MESOS-5656
> URL: https://issues.apache.org/jira/browse/MESOS-5656
> Project: Mesos
>  Issue Type: Bug
>  Components: build, cmake
>Affects Versions: 1.0.0
>Reporter: Benjamin Bannier
>Assignee: Andrew Schwartzmeyer
>  Labels: mesosphere
>
> The cmake build incompletely models dependencies on 3rdparty components. This 
> leads to incomplete and unusable build files generated for e.g., ninja.
> When generating a build file for ninja the build fails to start
> {code}
> % ninja
> ninja: error: 
> '3rdparty/zookeeper-3.4.8/src/zookeeper-3.4.8/src/c/lib/libzookeeper_mt.a', 
> needed by 'src/slave/mesos-agent', missing and no known rule to make it
> {code}
> An identical problem exists with leveldb (apparent after working around the 
> zookeeper dep issue)
> {code}
> % ninja
> ninja: error: '3rdparty/leveldb-1.4/src/leveldb-1.4/libleveldb.a', needed by 
> 'src/slave/mesos-agent', missing and no known rule to make it
> {code}
> The problem here is that a number of targets depend on library files produced 
> implicitly via some {{ExternalProject}} library (via {{AGENT_LIBS}}), but we 
> fail to declare rules for these targets (I could imagine: via e.g., 
> {{add_library}} with {{IMPORTED}}). This appears to be no problem for build 
> files generate for {{make}} as it doesn't require rules for all dependency 
> nodes.
> It appears that one should be able to use {{ExternalProject_Add}}'s 
> {{BUILD_BYPRODUCTS}}, e.g.,
> {code}
> ExternalProject_Add(
>${ZOOKEEPER_TARGET}
>BUILD_BYPRODUCTS  ${ZOOKEEPER_LIB}/lib/libzookeeper_mt.a
>PREFIX${ZOOKEEPER_CMAKE_ROOT}
>PATCH_COMMAND ${ZOOKEEPER_PATCH_CMD}
>CONFIGURE_COMMAND ${ZOOKEEPER_CONFIG_CMD}
>...
> {code}
> to declare rules for these files, but this was only added in cmake-3.2 while 
> we at least formally require only cmake-2.8,
> {code}
> cmake_minimum_required(VERSION 2.8)
> {code}
> {{git bisect}} points to the recent 
> {{6e199cc255cbf561fac575568b0594ac2b2c14f9}} for surfacing this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-6350) Raise minimum required cmake version

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu reassigned MESOS-6350:


Assignee: Andrew Schwartzmeyer

> Raise minimum required cmake version
> 
>
> Key: MESOS-6350
> URL: https://issues.apache.org/jira/browse/MESOS-6350
> Project: Mesos
>  Issue Type: Improvement
>  Components: cmake
>Reporter: Benjamin Bannier
>Assignee: Andrew Schwartzmeyer
>  Labels: mesosphere, microsoft, tech-debt
>
> We currently require at least cmake-2.8 which had its first point release 
> 2010 and last update 2013. Meanwhile upstream is preparing the release of 
> 3.7.0. While cmake support in Mesos is still experimental we should evaluate 
> how much we can increase the minimal required version so we are not locked 
> into an old version lacking desirable features.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-5656) Incomplete modelling of 3rdparty dependencies in cmake build

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-5656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu reassigned MESOS-5656:


Assignee: Andrew Schwartzmeyer

> Incomplete modelling of 3rdparty dependencies in cmake build
> 
>
> Key: MESOS-5656
> URL: https://issues.apache.org/jira/browse/MESOS-5656
> Project: Mesos
>  Issue Type: Bug
>  Components: build, cmake
>Affects Versions: 1.0.0
>Reporter: Benjamin Bannier
>Assignee: Andrew Schwartzmeyer
>  Labels: mesosphere
>
> The cmake build incompletely models dependencies on 3rdparty components. This 
> leads to incomplete and unusable build files generated for e.g., ninja.
> When generating a build file for ninja the build fails to start
> {code}
> % ninja
> ninja: error: 
> '3rdparty/zookeeper-3.4.8/src/zookeeper-3.4.8/src/c/lib/libzookeeper_mt.a', 
> needed by 'src/slave/mesos-agent', missing and no known rule to make it
> {code}
> An identical problem exists with leveldb (apparent after working around the 
> zookeeper dep issue)
> {code}
> % ninja
> ninja: error: '3rdparty/leveldb-1.4/src/leveldb-1.4/libleveldb.a', needed by 
> 'src/slave/mesos-agent', missing and no known rule to make it
> {code}
> The problem here is that a number of targets depend on library files produced 
> implicitly via some {{ExternalProject}} library (via {{AGENT_LIBS}}), but we 
> fail to declare rules for these targets (I could imagine: via e.g., 
> {{add_library}} with {{IMPORTED}}). This appears to be no problem for build 
> files generate for {{make}} as it doesn't require rules for all dependency 
> nodes.
> It appears that one should be able to use {{ExternalProject_Add}}'s 
> {{BUILD_BYPRODUCTS}}, e.g.,
> {code}
> ExternalProject_Add(
>${ZOOKEEPER_TARGET}
>BUILD_BYPRODUCTS  ${ZOOKEEPER_LIB}/lib/libzookeeper_mt.a
>PREFIX${ZOOKEEPER_CMAKE_ROOT}
>PATCH_COMMAND ${ZOOKEEPER_PATCH_CMD}
>CONFIGURE_COMMAND ${ZOOKEEPER_CONFIG_CMD}
>...
> {code}
> to declare rules for these files, but this was only added in cmake-3.2 while 
> we at least formally require only cmake-2.8,
> {code}
> cmake_minimum_required(VERSION 2.8)
> {code}
> {{git bisect}} points to the recent 
> {{6e199cc255cbf561fac575568b0594ac2b2c14f9}} for surfacing this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (MESOS-7866) CMake build system improvements

[Joseph Wu (JIRA)]

Joseph Wu created MESOS-7866:


 Summary: CMake build system improvements
 Key: MESOS-7866
 URL: https://issues.apache.org/jira/browse/MESOS-7866
 Project: Mesos
  Issue Type: Epic
  Components: build
Reporter: Joseph Wu
Assignee: Joseph Wu


This is a followup Epic for the [CMake 
epic|https://issues.apache.org/jira/browse/MESOS-898].

This includes items that will be tackled after we officially switch the build 
system from Autotools to CMake.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-3107) Define CMake style guide

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-3107:
-
Shepherd: Joseph Wu  (was: Benjamin Hindman)

> Define CMake style guide
> 
>
> Key: MESOS-3107
> URL: https://issues.apache.org/jira/browse/MESOS-3107
> Project: Mesos
>  Issue Type: Task
>  Components: cmake
>Reporter: Alex Clemmer
>Assignee: Andrew Schwartzmeyer
>  Labels: build, cmake
>
> The short story is that it is important to be principled about how the CMake 
> build system is maintained, because there CMake language makes it difficult 
> to statically verify that a configuration is correct. It is not unique in 
> this regard, but (make is arguably even worse) but it is something that's 
> important to make sure we get right.
> The longer story is, CMake's language is dynamically scoped and often has 
> somewhat odd defaults for variable values (_e.g._, IIRC, target names passed 
> to ExternalProject_Add default to "PREFIX" instead of erroring out). This 
> means that it is rare to get a configuration-time error (_i.e._, CMake 
> usually doesn't say something like "hey this variable isn't defined"), and in 
> large projects, this can make it very difficult to know where definitions 
> come from, or whether it's important that one config routine runs before 
> another. Dynamic scoping also makes it particularly easy to write spaghetti 
> code, which is clearly undesirable for something as important as a build 
> system.
> Thus, it is particularly important that we lay down our expectations for how 
> the CMake system is to be structured. This might include:
> * Function naming (_e.g._, making it easy to tell whether a function was 
> defined by us, and where it was defined; so we might say that we want our 
> functions to have an underscore to start, and start with the package the come 
> from, like libprocess, so that we know where to look for the definition.)
> * What assertions we want to check variable values against, so that we can 
> replace subtle errors (_e.g._, a library is accidentally named something 
> silly like "PREFIX.0.0.1") with an obvious ones (_e.g._, "You have failed to 
> define your target name, so CMake has defaulted to 'PREFIX'; please check 
> your configuration routines")
> * Decisions of what goes where. (_e.g._, the most complex parts of the CMake 
> MVPs is in the configuration routines, like `MesosConfigure.cmake`; to curb 
> this, we should have strict rules about what goes in that file vs other 
> files, and how we know what is to be run before what. Part of this should 
> probably be prominent comments explaining the structure of the project, so 
> that people aren't confused!)
> * And so on.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-898) Introduce CMake as an alternative build system.

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-898:

Labels: build cmake mesosphere  (was: build mesosphere)

> Introduce CMake as an alternative build system.
> ---
>
> Key: MESOS-898
> URL: https://issues.apache.org/jira/browse/MESOS-898
> Project: Mesos
>  Issue Type: Epic
>  Components: build
>Reporter: Timothy St. Clair
>Assignee: Joseph Wu
>  Labels: build, cmake, mesosphere
>
> This is a rather substantial undertaking, so I would want upstream 
> debate+buy-in prior to full commitment.  The basic premise is: upstream 
> rebundles several of its dependencies in part to tightly control its stack.  
> This is not out of the norm, but in order to be picked up by distribution 
> channels it needs to built against system dependencies, and rebundling is 
> strictly forbidden.  Given that the mesos primary target platform are 
> data-center distributions such as RHEL/CENTOS/SL it makes sense to still have 
> bundling support for those who do not have dependencies in their channels 
> "yet".  This is where cmake can be win with it's uber macros 
> (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).  
> I do not know of any equivalent in the autotools world, other then to brew 
> your own solution.   I've done this type of work in the past, and completely 
> transformed condor and would leverage a lot of the work that was done there. 
> I currently have a tracking branch where I've started this work, but before I 
> go off into the woods, it makes sense to have a debate in public. 
> The primary benefits are: 
> 1. Enable downstream channels to easily distro without carrying a large patch 
> sets. 
> 2. Still support existing "non-proper" distribution methods. 
> 3. Harden / future proof dependent interfaces. 
> Side Benefits: 
> Audit current build mechanics.  
>  - Presently the language specific binding are not installed.  (.py & .jar)
>  - make -jX currently fails 
>  - optionally look in arm support. 
> Costs:
> 1. Time
> 2. Potential temporary destabilization
> 3. Infrastructure around build+test may need to change.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-3105) Port flag generation logic from the autotools solution to CMake

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-3105:
-
Shepherd: Joseph Wu  (was: Benjamin Hindman)

> Port flag generation logic from the autotools solution to CMake
> ---
>
> Key: MESOS-3105
> URL: https://issues.apache.org/jira/browse/MESOS-3105
> Project: Mesos
>  Issue Type: Task
>  Components: cmake
>Reporter: Alex Clemmer
>Assignee: Andrew Schwartzmeyer
>  Labels: autotools, build, cmake
>
> One major barrier to widespread adoption of the CMake-based build system 
> (other than the fact that we haven't implemented it yet!) is that most of our 
> institutional knowledge of the quirks of how to build Mesos across many 
> platforms is tied up in files like `configure.ac`.
> Therefore, a "good" CMake-based build system will require us to go through 
> these files systematically and manually port this logic to CMake (as well as 
> testing it).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-2153) Add support for systemd journal for logging

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116928#comment-16116928
 ] 

Joseph Wu commented on MESOS-2153:
--

Try using this module if you'd like journald logging immediately:
https://github.com/dcos/dcos-mesos-modules/tree/master/journald

This issue is for considering whether we should support the module explicitly 
in Mesos (like we do for the Logrotate container logger module).

> Add support for systemd journal for logging
> ---
>
> Key: MESOS-2153
> URL: https://issues.apache.org/jira/browse/MESOS-2153
> Project: Mesos
>  Issue Type: Improvement
>  Components: agent, master
>Reporter: Alexander Rukletsov
>Priority: Minor
>  Labels: mesosphere
>
> We should be able to redirect master and slave logs to systemd journal on the 
> systems where it's available.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (MESOS-7858) Launching a nested container with namespace/pid isolation, with glibc < 2.25, may deadlock the LinuxLauncher and MesosContainerizer

[Joseph Wu (JIRA)]

Joseph Wu created MESOS-7858:


 Summary: Launching a nested container with namespace/pid 
isolation, with glibc < 2.25, may deadlock the LinuxLauncher and 
MesosContainerizer
 Key: MESOS-7858
 URL: https://issues.apache.org/jira/browse/MESOS-7858
 Project: Mesos
  Issue Type: Bug
  Components: containerization
Affects Versions: 1.3.0
Reporter: Joseph Wu


This bug in glibc (fixed in glibc 2.25) will sometimes cause a child process of 
a {{fork}} to {{assert}} incorrectly, if the parent enters a new pid namespace 
before forking: 
https://sourceware.org/bugzilla/show_bug.cgi?id=15392
https://sourceware.org/bugzilla/show_bug.cgi?id=21386

The LinuxLauncher code happens to do this when launching nested containers:
* The MesosContainerizer process launches a subprocess, with a customized 
{{ns::clone}} function as an argument.  The thread then basically waits for the 
launch to succeed and return a child PID: 
https://github.com/apache/mesos/blob/1.3.x/src/slave/containerizer/mesos/linux_launcher.cpp#L495
* A separate thread in the Mesos agent forks and then waits for the grandchild 
to report a PID: 
https://github.com/apache/mesos/blob/1.3.x/src/linux/ns.hpp#L453
* The child of the fork first enters the namespaces (including a pid namespace) 
and then forks a grandchild.  The child then calls {{waitpid}} on the 
grandchild: https://github.com/apache/mesos/blob/1.3.x/src/linux/ns.hpp#L555
* Due to the glibc bug, the grandchild sometimes never returns from the 
{{fork}} here: https://github.com/apache/mesos/blob/1.3.x/src/linux/ns.hpp#L540

According to the glibc bug, we can work around this by:
{quote}
The obvious solution is just to use clone() after setns() and never use fork() 
- and one can certainly patch both programs to do so. Nevertheless it would be 
nice to see if fork() also worked after setns(), especially since there is no 
inherent reason for it not to.
{quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-6566) The Docker executor should not leak task env variables in the Docker command cmd line.

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096869#comment-16096869
 ] 

Joseph Wu commented on MESOS-6566:
--

NOTE: We basically reversed this change in {{1.2.1}} because of MESOS-6951.

> The Docker executor should not leak task env variables in the Docker command 
> cmd line.
> --
>
> Key: MESOS-6566
> URL: https://issues.apache.org/jira/browse/MESOS-6566
> Project: Mesos
>  Issue Type: Bug
>  Components: docker, security
>Reporter: Gastón Kleiman
>Assignee: Till Toenshoff
> Fix For: 1.2.0
>
>
> Task environment variables are sensitive, as they might contain secrets.
> The Docker executor starts tasks by executing a {{docker run}} command, and 
> it includes the env variables in the cmd line of the docker command, exposing 
> them to all the users in the machine:
> {code}
> $ ./src/mesos-execute --command="sleep 200" --containerizer=docker 
> --docker_image=alpine --env='{"foo": "bar"}' --master=10.0.2.15:5050 
> --name=test
> $ ps aux | grep bar
> [...] docker -H unix:///var/run/docker.sock run [...] -e foo=bar [...] alpine 
> -c sleep 200
> $
> {code}
> The Docker executor could pass Docker the {{--env-file}} flag, pointing it to 
> a file with the environment variables.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7492) Introduce a daemon manager in the agent.

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-7492:
-
Shepherd: Jie Yu
  Sprint: Mesosphere Sprint 60
Story Points: 5

> Introduce a daemon manager in the agent.
> 
>
> Key: MESOS-7492
> URL: https://issues.apache.org/jira/browse/MESOS-7492
> Project: Mesos
>  Issue Type: Task
>Reporter: Jie Yu
>Assignee: Joseph Wu
>  Labels: mesosphere, storage
>
> Once we have standalone container support from the containerizer, we should 
> consider adding a daemon manager inside the agent. It'll be like 'monit', 
> 'upstart' or 'systemd', but with very limited functionalities. For instance, 
> as a start, the manager will simply always restart the daemons if the daemon 
> fails. It'll also try to cleanup unknown daemons.
> This feature will be used to manage CSI plugin containers on the agent.
> The daemon manager should have an interface allowing operators to "register" 
> a daemon with a name and a config of the daemon. The daemon manager is 
> responsible for restarting the daemon if it crashes until some one explicitly 
> "unregister" it. Some simple backoff and health check functionality should be 
> provided.
> We probably need a small design doc for this.
> {code}
> message DaemonConfig {
>   optional ContainerInfo container;
>   optional CommandInfo command;
>   optional uint32 poll_interval;
>   optional uint32 initial_delay;
>   optional CheckInfo check; // For health check.
> }
> class DaemonManager
> {
> public:
>   Future register(
> const ContainerID& containerId,
> const DaemonConfig& config;
>   Future unregister(const ContainerID& containerId);
>   Future ps();
>   Future status(const ContainerID& containerId);
> };
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-6390) Ensure Python support scripts are linted

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-6390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090878#comment-16090878
 ] 

Joseph Wu commented on MESOS-6390:
--

Another chunk: 
{code}
commit c0e51a8d729aa6b06d70fa18259354536bc59b43
Author: Armand Grillet 
Date:   Mon Jul 17 11:16:23 2017 -0700

Linted support/post-reviews.py.

This will allow us to use PyLint on the
entire support directory in the future.

Review: https://reviews.apache.org/r/60233/

commit a536d101a52e21b32cb584d4df9d468e8b76b6a2
Author: Armand Grillet 
Date:   Mon Jul 17 11:41:55 2017 -0700

Linted support/push-commits.py.

This will allow us to use PyLint on the
entire support directory in the future.

Review: https://reviews.apache.org/r/60234/

commit 92896871bdb9f02066a403da789cdb70e6e496e4
Author: Armand Grillet 
Date:   Mon Jul 17 16:33:56 2017 -0700

Linted support/verify-reviews.py.

This will allow us to use PyLint on the
entire support directory in the future.

Review: https://reviews.apache.org/r/60236/
{code}

> Ensure Python support scripts are linted
> 
>
> Key: MESOS-6390
> URL: https://issues.apache.org/jira/browse/MESOS-6390
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Benjamin Bannier
>Assignee: Armand Grillet
>  Labels: newbie, python
>
> Currently {{support/mesos-style.py}} does not lint files under {{support/}}. 
> This is mostly due to the fact that these scripts are too inconsistent 
> style-wise that they wouldn't even pass the linter now.
> We should clean up all Python scripts under {{support/}} so they pass the 
> Python linter, and activate that directory in the linter for future 
> additions. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (MESOS-7802) Push-commits.py support script is too lenient when determining reviews to close

[Joseph Wu (JIRA)]

Joseph Wu created MESOS-7802:


 Summary: Push-commits.py support script is too lenient when 
determining reviews to close
 Key: MESOS-7802
 URL: https://issues.apache.org/jira/browse/MESOS-7802
 Project: Mesos
  Issue Type: Bug
Reporter: Joseph Wu
Priority: Minor


The support script {{support/push-commits.py}} can be used by committers to 
push commits and simultaneously close reviews.  However, it is currently quite 
easy to trick the script into closing unrelated reviews.

For example, if you have a commit message like:
{code}
Referring to multiple reviews in one commit message.

Review: https://reviews.apache.org/r/1/
Review: https://reviews.apache.org/r/2/
Review: https://reviews.apache.org/r/3/
Review: https://reviews.apache.org/r/4/
{code}

The script will do this:
{code}
$ support/push-commits.py --dry-run
Found reviews ['1', '2', '3', '4']
Pushing commits to apache
Closing review 1
Closing review 2
Closing review 3
Closing review 4
{code}

It is possible for this to happen non-maliciously, if the contributor's review 
description merely refers to another review in the same format.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7753) `log.LearnedMessage` could be rejected due to being sent from '@0.0.0.0:0'

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-7753:
-
Shepherd: Joseph Wu

> `log.LearnedMessage` could be rejected due to being sent from '@0.0.0.0:0'
> --
>
> Key: MESOS-7753
> URL: https://issues.apache.org/jira/browse/MESOS-7753
> Project: Mesos
>  Issue Type: Bug
>  Components: replicated log
>Reporter: Yan Xu
>
> This is due to the use of 
> https://github.com/apache/mesos/blob/ced7d69767142c912db0c2e01b95c0f5de791bd9/src/log/network.hpp#L247
>  which sets the message's {{from}} field to an empty UPID.
> The rationale for using this form is that there's no intended response for 
> this message. However with MESOS-7401, this message could be rejected.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7796) LIBPROCESS_IP isn't passed on to the fetcher

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088173#comment-16088173
 ] 

Joseph Wu commented on MESOS-7796:
--

Note: The logger modules just hard-code this:
{{environment.emplace("LIBPROCESS_IP", "127.0.0.1");}}

> LIBPROCESS_IP isn't passed on to the fetcher
> 
>
> Key: MESOS-7796
> URL: https://issues.apache.org/jira/browse/MESOS-7796
> Project: Mesos
>  Issue Type: Bug
>  Components: containerization, fetcher, network
>Affects Versions: 1.2.0, 1.2.1
>Reporter: Gastón Kleiman
>  Labels: mesosphere
>
> {{LIBPROCESS_IP}} is not passed on to the fetcher.
> The fetcher program uses libprocess, which depending on the DNS configuration 
> might fail during initialization:
> {noformat}
> I0710 17:40:36.866606  8157 fetcher.cpp:531] Fetcher Info: 
> {"cache_directory":"\/tmp\/mesos\/fetch\/slaves\/1f5cf498-82db-4976-8ba1-f2be90af3496-S1\/nobody","items":[{"action":"BYPASS_CACHE","uri":{"cache":false,"executable":false,"extract":true,"value":"REDACTED"}},{"action":"BYPASS_CACHE","uri":{"cache":false,"executable":false,"extract":true,"value":"REDACTED"}},{"action":"BYPASS_CACHE","uri":{"cache":false,"executable":false,"extract":true,"value":"REDACTED"}}],"sandbox_directory":"\/var\/lib\/mesos\/slave\/slaves\/1f5cf498-82db-4976-8ba1-f2be90af3496-S1\/frameworks\/1f5cf498-82db-4976-8ba1-f2be90af3496-\/executors\/cassandra.e1e042ef-6596-11e7-adc9-a6ba31bb9f5f\/runs\/5728acc2-33fc-4c39-ba7c-55908cbe5862","user":"nobody"}
> I0710 17:40:36.869302  8157 fetcher.cpp:442] Fetching URI 'REDACTED'
> I0710 17:40:36.869319  8157 fetcher.cpp:283] Fetching directly into the 
> sandbox directory
> I0710 17:40:36.869343  8157 fetcher.cpp:220] Fetching URI 'REDACTED'
> I0710 17:40:36.869359  8157 fetcher.cpp:163] Downloading resource from 
> 'REDACTED' to 
> '/var/lib/mesos/slave/slaves/1f5cf498-82db-4976-8ba1-f2be90af3496-S1/frameworks/1f5cf498-82db-4976-8ba1-f2be90af3496-/executors/cassandra.e1e042ef-6596-11e7-adc9-a6ba31bb9f5f/runs/5728acc2-33fc-4c39-ba7c-55908cbe5862/REDACTED'
> Failed to obtain the IP address for 
> 'ip-10-0-5-78.us-west-2.compute.internal'; the DNS service may not be able to 
> resolve it: Name or service not known
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7793) Agent reports 200 OK when its not

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087711#comment-16087711
 ] 

Joseph Wu commented on MESOS-7793:
--

The two {{/health}} endpoints are somewhat unfortunately named, as they do not 
actually give you any information except that libprocess has initialized.

{{/metrics/snapshot}} is generally a better choice, but that will require you 
to parse and filter the result.

> Agent reports 200 OK when its not
> -
>
> Key: MESOS-7793
> URL: https://issues.apache.org/jira/browse/MESOS-7793
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 1.0.0
> Environment: Centos 7.3.
> Master 1.0.1.
> Agent 1.0.0
>Reporter: Jamie Briant
> Attachments: mesos-agent-logs.txt
>
>
> Had an agent go offline and then fail to recover. It eventually entered a 
> state where it was not registered with the master, but where it neither 
> reported additional errors, nor existed. In this state, the agent reported 
> 200 OK on :5051/health



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-3394) Change glog download target for Windows when pull req is moved upstream

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-3394:
-
Shepherd: Joseph Wu  (was: Joris Van Remoortere)

> Change glog download target for Windows when pull req is moved upstream
> ---
>
> Key: MESOS-3394
> URL: https://issues.apache.org/jira/browse/MESOS-3394
> Project: Mesos
>  Issue Type: Task
>  Components: cmake
>Reporter: Alex Clemmer
>Assignee: Andrew Schwartzmeyer
>  Labels: build, cmake, mesosphere
>
> To build on Windows, we have to build glog on Windows. But, glog doesn't 
> build on Windows, so we had to submit a patch to the project. So, to build on 
> Windows, we download the patched version directly from the pull request that 
> was sent to the glog repository on GitHub.
> When these patches move upstream, we need to change this to point at the 
> "real" glog release instead of the pull request.
> (For details see the `CMakeLists.txt` in `3rdparty/libprocess/3rdparty`.)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7548) net::hostname takes 5 seconds on Windows

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081332#comment-16081332
 ] 

Joseph Wu commented on MESOS-7548:
--

Related review: https://reviews.apache.org/r/60291/

> net::hostname takes 5 seconds on Windows
> 
>
> Key: MESOS-7548
> URL: https://issues.apache.org/jira/browse/MESOS-7548
> Project: Mesos
>  Issue Type: Bug
> Environment: Windows 10
>Reporter: Andrew Schwartzmeyer
>Assignee: Andrew Schwartzmeyer
>  Labels: stout, windows
>
> We've determined that when starting the master in a unit test, instead of 
> milliseconds, it takes upwards of five seconds to start. However, providing a 
> set of flags where `hostname` is specified, it takes less than a second. 
> We've determined that the `net::hostname` call on Windows is taking an 
> inordinate amount of time, and this is problematic when every instantiation 
> of a master or agent uses it, especially across hundreds of tests.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-7751) Mesos failed to build on Windows due to error C2039: 'parse': is not a member of 'mesos::internal::protobuf'

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu reassigned MESOS-7751:


   Resolution: Fixed
 Assignee: Jan Schlicht
Fix Version/s: 1.4.0

Fixed in review: https://reviews.apache.org/r/60653/

> Mesos failed to build on Windows due to error C2039: 'parse': is not a member 
> of 'mesos::internal::protobuf'
> 
>
> Key: MESOS-7751
> URL: https://issues.apache.org/jira/browse/MESOS-7751
> Project: Mesos
>  Issue Type: Bug
>  Components: build
> Environment: Windows Server 2012 R2 + VS2015 Update 3 + Mesos master 
> branch latest source
>Reporter: Karen Huang
>Assignee: Jan Schlicht
> Fix For: 1.4.0
>
>
> I try to build mesos with Debug|x64 configuration on Windows. It failed to 
> build due error C2039: 'parse': is not a member of 
> 'mesos::internal::protobuf'. This issue can be reproduced from master branch 
> revision 15078ad 
> [15078ad|https://github.com/apache/mesos/commit/15078adddec1a2fca80978eb29b6889eb15db4b1#diff-6141d9940e12f6e495954cacbe037016].
>  Could you please take a look at this? Thanks in advance!
> Here is repro steps:
>  1. git clone -c core.autocrlf=true https://github.com/apache/mesos 
> D:\mesos\src
>  2. Open a VS amd64 command prompt as admin and browse to D:\mesos\src
>  3. bootstrap.bat
>  4. mkdir build_x64 && pushd build_x64
>  5. cmake ..\src -G "Visual Studio 14 2015 Win64" 
> -DCMAKE_SYSTEM_VERSION=10.0.14393.0 -DENABLE_LIBEVENT=1 
> -DHAS_AUTHENTICATION=0 -DPATCHEXE_PATH=$quotC:\gnuwin32\bin$quot -T host=x64 
> 6. msbuild Mesos.sln /p:Configuration=Debug /p:Platform=x64 /m /t:Rebuild
> Acutal result:
> "D:\Mesos\build_x64\Mesos.sln" (Rebuild target) (1) ->
>"D:\Mesos\build_x64\src\mesos-1.4.0.vcxproj.metaproj" (Rebuild target) 
> (16) ->
>"D:\Mesos\build_x64\src\mesos-1.4.0.vcxproj" (Rebuild target) (57) ->
>(ClCompile target) -> 
>  D:\Mesos\src\src\resource_provider\daemon.cpp(130): error C2039: 
> 'parse': is not a member of 'mesos::internal::protobuf' 
> [D:\Mesos\build_x64\src\mesos-1.4.0.vcxproj]
>  D:\Mesos\src\src\resource_provider\daemon.cpp(130): error C2065: 
> 'parse': undeclared identifier [D:\Mesos\build_x64\src\mesos-1.4.0.vcxproj]
>  D:\Mesos\src\src\resource_provider\daemon.cpp(130): error C2275: 
> 'mesos::ResourceProviderInfo': illegal use of this type as an expression 
> [D:\Mesos\build_x64\src\mesos-1.4.0.vcxproj]
>  D:\Mesos\src\src\resource_provider\daemon.cpp(129): error C2512: 
> 'Try': no appropriate default constructor 
> available [D:\Mesos\build_x64\src\mesos-1.4.0.vcxproj]
> 387 Warning(s)
> 4 Error(s)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-6390) Ensure Python support scripts are linted

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-6390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058548#comment-16058548
 ] 

Joseph Wu commented on MESOS-6390:
--

Here's a patch to enable linting on the support directory:
https://reviews.apache.org/r/60353/

It should go in after we've updated the style of existing scripts.

> Ensure Python support scripts are linted
> 
>
> Key: MESOS-6390
> URL: https://issues.apache.org/jira/browse/MESOS-6390
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Benjamin Bannier
>Assignee: Armand Grillet
>  Labels: newbie, python
>
> Currently {{support/mesos-style.py}} does not lint files under {{support/}}. 
> This is mostly due to the fact that these scripts are too inconsistent 
> style-wise that they wouldn't even pass the linter now.
> We should clean up all Python scripts under {{support/}} so they pass the 
> Python linter, and activate that directory in the linter for future 
> additions. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-6390) Ensure Python support scripts are linted

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-6390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058479#comment-16058479
 ] 

Joseph Wu commented on MESOS-6390:
--

Committed a chunk of the linting changes:
{code}
commit f030c4bd5e8f3cae780052fe62dc260dd7e6eac9
Author: Armand Grillet 
Date:   Wed Jun 21 14:54:31 2017 -0700

Linted support/mesos-style.py.

This will allow us to use PyLint on the
entire support directory in the future.

Review: https://reviews.apache.org/r/60227/

commit 09fc35978c77ed0f22df1823f5c9220f1ed098ed
Author: Armand Grillet 
Date:   Wed Jun 21 15:00:52 2017 -0700

Linted support/apply-reviews.py.

This will allow us to use PyLint on the
entire support directory in the future.

Review: https://reviews.apache.org/r/60228/

commit bb8e738b8e10a8061318d081cf75de2e0d0bc47c
Author: Armand Grillet 
Date:   Wed Jun 21 15:33:50 2017 -0700

Linted support/generate-endpoint-help.py.

This will allow us to use PyLint on the
entire support directory in the future.

Review: https://reviews.apache.org/r/60229/

commit 635b3ed52c51bfad2d5d1e3ad76a7454a7552baf
Author: Armand Grillet 
Date:   Wed Jun 21 16:18:32 2017 -0700

Linted support/jsonurl.py.

This will allow us to use PyLint on the
entire support directory in the future.

Review: https://reviews.apache.org/r/60230/

commit 62de760059cd07ea0127a6b58588dd777abfa4d8
Author: Armand Grillet 
Date:   Wed Jun 21 16:32:16 2017 -0700

Linted support/mesos-gtest-runner.py.

This will allow us to use PyLint on the
entire support directory in the future.

Review: https://reviews.apache.org/r/60231/

commit 95ae610acd5cdd52f561776f9bceb727d8239abc
Author: Armand Grillet 
Date:   Wed Jun 21 16:34:11 2017 -0700

Linted support/mesos-split.py.

This will allow us to use PyLint on the
entire support directory in the future.

Review: https://reviews.apache.org/r/60232/
{code}

> Ensure Python support scripts are linted
> 
>
> Key: MESOS-6390
> URL: https://issues.apache.org/jira/browse/MESOS-6390
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Benjamin Bannier
>Assignee: Armand Grillet
>  Labels: newbie, python
>
> Currently {{support/mesos-style.py}} does not lint files under {{support/}}. 
> This is mostly due to the fact that these scripts are too inconsistent 
> style-wise that they wouldn't even pass the linter now.
> We should clean up all Python scripts under {{support/}} so they pass the 
> Python linter, and activate that directory in the linter for future 
> additions. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7302) Support launching standalone containers.

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057970#comment-16057970
 ] 

Joseph Wu commented on MESOS-7302:
--

API hasn't been proposed yet.  But it will look very similar to the 
{{LAUNCH_NESTED_CONTAINER}} call; minus a {{ContainerID}} and plus a 
{{Resources}} field.

> Support launching standalone containers.
> 
>
> Key: MESOS-7302
> URL: https://issues.apache.org/jira/browse/MESOS-7302
> Project: Mesos
>  Issue Type: Epic
>  Components: containerization
>Reporter: Jie Yu
>Assignee: Joseph Wu
>  Labels: mesosphere
>
> Containerizer should support launching containers (both top level and nested) 
> that are not tied to a particular Mesos task or executor. This is for the 
> case where the agent wants to launch some system containers (e.g., for CSI 
> plugin) that will be managed by Mesos.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7302) Support launching standalone containers.

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057946#comment-16057946
 ] 

Joseph Wu commented on MESOS-7302:
--

For the MVP, containers launched via the (to be implemented) standalone 
container API will be isolated like any other container.  But the Master will 
not know about them.  So if you abuse the Operator API, it would be possible to 
oversubscribe the Agent.

I'd need to add some additional plumbing from the Agent to the Master's 
allocator in order to properly account for these resources (albeit racily, as 
standalone containers are launched outside of the offer cycle).

> Support launching standalone containers.
> 
>
> Key: MESOS-7302
> URL: https://issues.apache.org/jira/browse/MESOS-7302
> Project: Mesos
>  Issue Type: Epic
>  Components: containerization
>Reporter: Jie Yu
>Assignee: Joseph Wu
>  Labels: mesosphere
>
> Containerizer should support launching containers (both top level and nested) 
> that are not tied to a particular Mesos task or executor. This is for the 
> case where the agent wants to launch some system containers (e.g., for CSI 
> plugin) that will be managed by Mesos.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7157) Support the Release build configuration on Windows

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-7157:
-
 Labels: mesosphere windows  (was: )
Summary: Support the Release build configuration on Windows  (was: LINK : 
fatal error LNK1181: cannot open input file 'glog.lib' when build mesos on 
Release configuration on Windows)

> Support the Release build configuration on Windows
> --
>
> Key: MESOS-7157
> URL: https://issues.apache.org/jira/browse/MESOS-7157
> Project: Mesos
>  Issue Type: Bug
>  Components: build
> Environment: Windows10 + 64-bit + Visual studio 2015 
>Reporter: PhoebeHui
>Assignee: Andrew Schwartzmeyer
>Priority: Minor
>  Labels: mesosphere, windows
>
> I try to build Mesos with vs2015 on Release configuration on Windows, but it 
> failed with link error LNK1181. It seems that it hard code the glog.lib path:
> if I set Configuration=Debug, it will find glog.lib on
> D:/mesos/build_x64/3rdparty/glog-0.3.4/src/glog-0.3.4-build/Debug or
> D:/mesos/build_x64/3rdparty/glog-0.3.4/src/glog-0.3.4-build/Debug/Debug
> it works.
> if I set Configuration=Release, it will find glog.lib on 
> D:/mesos/build_x64/3rdparty/glog-0.3.4/src/glog-0.3.4-build/Debug or
> D:/mesos/build_x64/3rdparty/glog-0.3.4/src/glog-0.3.4-build/Debug/Release
> it doesn't work, since 
> D:/mesos/build_x64/3rdparty/glog-0.3.4/src/glog-0.3.4-build/Debug doesn't 
> exsit. the glog.lib located on  
> D:/mesos/build_x64/3rdparty/glog-0.3.4/src/glog-0.3.4-build/Release
> Repro steps:
> Get source: 
> git clone -c core.autocrlf=true https://github.com/apache/mesos D:\mesos\src
> Build the project:
> cd d:\mesos\src
> .\bootstrap.bat 
> cd..
>  mkdir build_x64 && pushd build_x64
> cmake ..\src -G "Visual Studio 14 2015 Win64" -DENABLE_LIBEVENT=1 
> -DHAS_AUTHENTICATION=0 -DPATCHEXE_PATH="C:\Program Files (x86)\GnuWin32\bin" 
> msbuild Mesos.sln /p:Configuration=Release /p:PreferredToolArchitecture=x64 
> /m 
> It will build failed with Link error:
> "F:\mesos\build_x64\Mesos.sln" (Rebuild target) (1) ->
>"D:\mesos\build_x64\3rdparty\stout\tests\stout-tests.vcxproj.metaproj" 
> (Rebuild target) (37) ->
>"D:\mesos\build_x64\3rdparty\stout\tests\stout-tests.vcxproj" (Rebuild 
> target) (52) ->
>(Link target) -> 
>  LINK : fatal error LNK1181: cannot open input file 'glog.lib' 
> [D:\mesos\build_x64\3rdparty\stout\tests\stout-tests.vcxproj]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7524) Basic fetcher success metrics

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-7524:
-
Shepherd: Joseph Wu

> Basic fetcher success metrics
> -
>
> Key: MESOS-7524
> URL: https://issues.apache.org/jira/browse/MESOS-7524
> Project: Mesos
>  Issue Type: Bug
>  Components: fetcher
>Reporter: James Peach
>Assignee: James Peach
>
> There are no metrics for the fetcher. As minimum we should have counters for:
> * successful fetcher invocations
> * failed fetcher invocations
> It would also be useful to know the fetch time, though that could be highly 
> variable depending on the cluster usage.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7305) Adjust the recover logic of MesosContainerizer to allow standalone containers.

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-7305:
-
  Sprint: Mesosphere Sprint 57
Story Points: 5
  Labels: mesosphere  (was: )

> Adjust the recover logic of MesosContainerizer to allow standalone containers.
> --
>
> Key: MESOS-7305
> URL: https://issues.apache.org/jira/browse/MESOS-7305
> Project: Mesos
>  Issue Type: Task
>  Components: containerization
>Reporter: Jie Yu
>Assignee: Joseph Wu
>  Labels: mesosphere
>
> The current recovery logic in MesosContainerizer assumes that all top level 
> containers are tied to some Mesos executors. Adding standalone containers 
> will invalid this assumption. The recovery logic must be changed to adapt to 
> that.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MESOS-7604) SlaveTest.ExecutorReregistrationTimeoutFlag aborts on Windows

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033972#comment-16033972
 ] 

Joseph Wu commented on MESOS-7604:
--

The agent sets the {{id}} field of the two AgentInfos to be equal prior to 
comparison.  But the output from the aborted test indicates that the two 
{{id}}s are not equal.  Temporarily disabling the test pending further 
investigation:
{code}
commit 6fa67899ef0f06dc6a0fdbb65f5efe9a7e12e76e
Author: Joseph Wu 
Date:   Thu Jun 1 17:32:06 2017 -0700

Windows: Disabled SlaveTest.ExecutorReregistrationTimeoutFlag.

This test aborts on Windows due to a difference in AgentID during
agent recovery.
{code}

> SlaveTest.ExecutorReregistrationTimeoutFlag aborts on Windows
> -
>
> Key: MESOS-7604
> URL: https://issues.apache.org/jira/browse/MESOS-7604
> Project: Mesos
>  Issue Type: Bug
>  Components: test
>Affects Versions: 1.4.0
> Environment: Windows
>Reporter: Joseph Wu
>  Labels: mesosphere, windows
>
> {code}
> [ RUN  ] SlaveTest.ExecutorReregistrationTimeoutFlag
> rk ae9679b1-67c9-4db6-8187-0641b0e929d2-
> I0601 23:53:23.488337  2748 master.cpp:1156] Master terminating
> I0601 23:53:23.492337  2728 hierarchical.cpp:579] Removed agent 
> ae9679b1-67c9-4db6-8187-0641b0e929d2-S0
> I0601 23:53:23.530340  1512 cluster.cpp:162] Creating default 'local' 
> authorizer
> I0601 23:53:23.544342  2728 master.cpp:436] Master 
> f07f4fdd-cd91-4d62-bf33-169b20d02020 (ip-172-20-128-1.ec2.internal) started 
> on 172.20.128.1:51241
> I0601 23:53:23.545341  2728 master.cpp:438] Flags at startup: --acls="" 
> --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" 
> --allocation_interval="1secs" --allocator="HierarchicalDRF" 
> --authenticate_agents="false" --authenticate_frameworks="false" 
> --authenticate_http_frameworks="true" --authenticate_http_readonly="true" 
> --authenticate_http_readwrite="true" --authenticators="crammd5" 
> --authorizers="local" --credentials="C:\temp\FWZORI\credentials" 
> --filter_gpu_resources="true" --framework_sorter="drf" --help="false" 
> --hostname_lookup="true" --http_authenticators="basic" 
> --http_framework_authenticators="basic" --initialize_driver_logging="true" 
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" 
> --max_agent_ping_timeouts="5" --max_completed_frameworks="50" 
> --max_completed_tasks_per_framework="1000" 
> --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false" 
> --recovery_agent_removal_limit="100%" --registry="in_memory" 
> --registry_fetch_timeout="1mins" --registry_gc_interval="15mins" 
> --registry_max_agent_age="2weeks" --registry_max_agent_count="102400" 
> --registry_store_timeout="100secs" --registry_strict="false" 
> --root_submissions="true" --user_sorter="drf" --version="false" 
> --webui_dir="/webui" --work_dir="C:\temp\FWZORI\master" 
> --zk_session_timeout="10secs"
> I0601 23:53:23.550338  2728 master.cpp:515] Master only allowing 
> authenticated HTTP frameworks to register
> I0601 23:53:23.550338  2728 credentials.hpp:37] Loading credentials for 
> authentication from 'C:\temp\FWZORI\credentials'
> I0601 23:53:23.552338  2728 http.cpp:975] Creating default 'basic' HTTP 
> authenticator for realm 'mesos-master-readonly'
> I0601 23:53:23.553339  2728 http.cpp:975] Creating default 'basic' HTTP 
> authenticator for realm 'mesos-master-readwrite'
> I0601 23:53:23.554340  2728 http.cpp:975] Creating default 'basic' HTTP 
> authenticator for realm 'mesos-master-scheduler'
> I0601 23:53:23.555341  2728 master.cpp:640] Authorization enabled
> I0601 23:53:23.570340  2124 master.cpp:2159] Elected as the leading master!
> I0601 23:53:23.570340  2124 master.cpp:1698] Recovering from registrar
> I0601 23:53:23.573341  1920 registrar.cpp:389] Successfully fetched the 
> registry (0B) in 0ns
> I0601 23:53:23.573341  1920 registrar.cpp:493] Applied 1 operations in 0ns; 
> attempting to update the registry
> I0601 23:53:23.575342  1920 registrar.cpp:550] Successfully updated the 
> registry in 0ns
> I0601 23:53:23.576344  1920 registrar.cpp:422] Successfully recovered 
> registrar
> I0601 23:53:23.577342  2728 master.cpp:1797] Recovered 0 agents from the 
> registry (167B); allowing 10mins for agents to re-register
> I0601 23:53:23.595341  1512 containerizer.cpp:230] Using isolation: 
> windows/cpu,filesystem/windows,environment_secret
> I0601 23:53:23.596343  1512 provisioner.cpp:255] Using default backend 'copy'
> I0601 23:53:23.626343  3976 slave.cpp:248] Mesos agent started on 
> (133)@172.20.128.1:51241
> I0601 23:53:23.627342  3976 slave.cpp:249] Flags at startup: 
> --appc_simple_discovery_uri_prefix="http://; 
> --appc_store_dir="C:\temp\kglZbS\store\appc" 
> 

[jira] [Created] (MESOS-7604) SlaveTest.ExecutorReregistrationTimeoutFlag aborts on Windows

[Joseph Wu (JIRA)]

Joseph Wu created MESOS-7604:


 Summary: SlaveTest.ExecutorReregistrationTimeoutFlag aborts on 
Windows
 Key: MESOS-7604
 URL: https://issues.apache.org/jira/browse/MESOS-7604
 Project: Mesos
  Issue Type: Bug
  Components: test
Affects Versions: 1.4.0
 Environment: Windows
Reporter: Joseph Wu


{code}
[ RUN  ] SlaveTest.ExecutorReregistrationTimeoutFlag
rk ae9679b1-67c9-4db6-8187-0641b0e929d2-
I0601 23:53:23.488337  2748 master.cpp:1156] Master terminating
I0601 23:53:23.492337  2728 hierarchical.cpp:579] Removed agent 
ae9679b1-67c9-4db6-8187-0641b0e929d2-S0
I0601 23:53:23.530340  1512 cluster.cpp:162] Creating default 'local' authorizer
I0601 23:53:23.544342  2728 master.cpp:436] Master 
f07f4fdd-cd91-4d62-bf33-169b20d02020 (ip-172-20-128-1.ec2.internal) started on 
172.20.128.1:51241
I0601 23:53:23.545341  2728 master.cpp:438] Flags at startup: --acls="" 
--agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" 
--allocation_interval="1secs" --allocator="HierarchicalDRF" 
--authenticate_agents="false" --authenticate_frameworks="false" 
--authenticate_http_frameworks="true" --authenticate_http_readonly="true" 
--authenticate_http_readwrite="true" --authenticators="crammd5" 
--authorizers="local" --credentials="C:\temp\FWZORI\credentials" 
--filter_gpu_resources="true" --framework_sorter="drf" --help="false" 
--hostname_lookup="true" --http_authenticators="basic" 
--http_framework_authenticators="basic" --initialize_driver_logging="true" 
--log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" 
--max_agent_ping_timeouts="5" --max_completed_frameworks="50" 
--max_completed_tasks_per_framework="1000" 
--max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false" 
--recovery_agent_removal_limit="100%" --registry="in_memory" 
--registry_fetch_timeout="1mins" --registry_gc_interval="15mins" 
--registry_max_agent_age="2weeks" --registry_max_agent_count="102400" 
--registry_store_timeout="100secs" --registry_strict="false" 
--root_submissions="true" --user_sorter="drf" --version="false" 
--webui_dir="/webui" --work_dir="C:\temp\FWZORI\master" 
--zk_session_timeout="10secs"
I0601 23:53:23.550338  2728 master.cpp:515] Master only allowing authenticated 
HTTP frameworks to register
I0601 23:53:23.550338  2728 credentials.hpp:37] Loading credentials for 
authentication from 'C:\temp\FWZORI\credentials'
I0601 23:53:23.552338  2728 http.cpp:975] Creating default 'basic' HTTP 
authenticator for realm 'mesos-master-readonly'
I0601 23:53:23.553339  2728 http.cpp:975] Creating default 'basic' HTTP 
authenticator for realm 'mesos-master-readwrite'
I0601 23:53:23.554340  2728 http.cpp:975] Creating default 'basic' HTTP 
authenticator for realm 'mesos-master-scheduler'
I0601 23:53:23.555341  2728 master.cpp:640] Authorization enabled
I0601 23:53:23.570340  2124 master.cpp:2159] Elected as the leading master!
I0601 23:53:23.570340  2124 master.cpp:1698] Recovering from registrar
I0601 23:53:23.573341  1920 registrar.cpp:389] Successfully fetched the 
registry (0B) in 0ns
I0601 23:53:23.573341  1920 registrar.cpp:493] Applied 1 operations in 0ns; 
attempting to update the registry
I0601 23:53:23.575342  1920 registrar.cpp:550] Successfully updated the 
registry in 0ns
I0601 23:53:23.576344  1920 registrar.cpp:422] Successfully recovered registrar
I0601 23:53:23.577342  2728 master.cpp:1797] Recovered 0 agents from the 
registry (167B); allowing 10mins for agents to re-register
I0601 23:53:23.595341  1512 containerizer.cpp:230] Using isolation: 
windows/cpu,filesystem/windows,environment_secret
I0601 23:53:23.596343  1512 provisioner.cpp:255] Using default backend 'copy'
I0601 23:53:23.626343  3976 slave.cpp:248] Mesos agent started on 
(133)@172.20.128.1:51241
I0601 23:53:23.627342  3976 slave.cpp:249] Flags at startup: 
--appc_simple_discovery_uri_prefix="http://; 
--appc_store_dir="C:\temp\kglZbS\store\appc" 
--authenticate_http_readonly="true" --authenticate_http_readwrite="true" 
--authenticatee="crammd5" --authentication_backoff_factor="1secs" 
--authorizer="local" --container_disk_watch_interval="15secs" 
--containerizers="mesos" --default_role="*" --disk_watch_interval="1mins" 
--docker="docker" --docker_kill_orphans="true" 
--docker_registry="https://registry-1.docker.io; --docker_remove_delay="6hrs" 
--docker_socket="//./pipe/docker_engine" --docker_stop_timeout="0ns" 
--docker_store_dir="C:\temp\kglZbS\store\docker" 
--docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" 
--enforce_container_disk_quota="false" --executor_registration_timeout="1mins" 
--executor_reregistration_timeout="15secs" 
--executor_shutdown_grace_period="5secs" 
--fetcher_cache_dir="C:\temp\kglZbS\fetch" --fetcher_cache_size="2GB" 
--frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" 
--hadoop_home="" --help="false" --hostname_lookup="true" 

[jira] [Updated] (MESOS-7597) libprocess build is broken

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-7597:
-
Shepherd: Joseph Wu

Fix: https://reviews.apache.org/r/59691/

> libprocess build is broken
> --
>
> Key: MESOS-7597
> URL: https://issues.apache.org/jira/browse/MESOS-7597
> Project: Mesos
>  Issue Type: Bug
>  Components: libprocess
> Environment: Windows
>Reporter: Andrew Schwartzmeyer
>Assignee: Andrew Schwartzmeyer
>Priority: Critical
>
> Commit 8fbbebfb6 broke the build:
> C:\Users\andschwa\src\mesos\3rdparty\libprocess\src\process.cpp(2877): error 
> C2882: 'flags': illegal use of namespace identifier in expressio
> This is probably due to the use of pre-compiled headers on Windows (`flags` 
> as an identifier has been problematic).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MESOS-7342) Port Docker tests

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16025642#comment-16025642
 ] 

Joseph Wu commented on MESOS-7342:
--

{code}
commit c4df8f7b0e48e190111ef427080bf3ca5019a25a
Author: John Kordich 
Date:   Thu May 25 17:24:10 2017 -0700

Windows: Enabled DOCKER and ROOT test filters.

This flips two test filters on Windows, so that all Docker and Root
tests are now enabled by default on Windows (rather than disabled).

On Windows, everything must be run as Administrator (like root, but
also somewhat different), so all ROOT tests are enabled by default.
The DOCKER filter now has the same behavior on Windows and Linux.

However, since many of the tests affected by these filters are
still not working on Windows, some individual tests have been
disabled.

The main test enabled by this commit is:
`DockerTest.ROOT_DOCKER_Version`.

Review: https://reviews.apache.org/r/59353/
{code}

> Port Docker tests
> -
>
> Key: MESOS-7342
> URL: https://issues.apache.org/jira/browse/MESOS-7342
> Project: Mesos
>  Issue Type: Bug
>  Components: docker
> Environment: Windows 10
>Reporter: Andrew Schwartzmeyer
>Assignee: John Kordich
>  Labels: microsoft, windows
>
> While one of Daniel Pravat's last acts was introducing the the Docker 
> containerizer for Windows, we don't have tests. We need to port 
> `docker_tests.cpp` and `docker_containerizer_tests.cpp` to Windows.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MESOS-7565) Container with "Contiv" networking fails upon startup

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16025439#comment-16025439
 ] 

Joseph Wu commented on MESOS-7565:
--

It appears that, with your networking setup, the executor cannot open a 
connection to the agent (presumably located at {{192.168.110.2}}).  That's why 
the executor logs {{Agent exited ... shutting down}}.

> Container with "Contiv" networking fails upon startup
> -
>
> Key: MESOS-7565
> URL: https://issues.apache.org/jira/browse/MESOS-7565
> Project: Mesos
>  Issue Type: Bug
>  Components: network
>Affects Versions: 1.2.2, 1.3.1
> Environment: centos 7.3
>Reporter: Hao Yixin
>
> When launching a task through Marathon and asking the task to assign an IP 
> (using Contiv networking):
> Log from mesos-slave:
> I0525 18:52:15.898908  1210 linux_launcher.cpp:429] Launching container 
> c4b299e6-629a-4a99-bd88-cfbca0262b1a and cloning with namespaces CLONE_NEWNS 
> | CLONE_NEWUTS | CLONE_NEWNET
> I0525 18:52:15.900668  1210 systemd.cpp:96] Assigned child process '3985' to 
> 'mesos_executors.slice'
> I0525 18:52:15.902612  1206 containerizer.cpp:1623] Checkpointing container's 
> forked pid 3985 to 
> '/var/lib/mesos/meta/slaves/00e6894c-d896-4a3d-8e79-679077f2af81-S4/frameworks/00e6894c-d896-4a3d-8e79-679077f2af81-/executors/container.1467.373c1d9b-4138-11e7-9117-024221dd5669/runs/c4b299e6-629a-4a99-bd88-cfbca0262b1a/pids/forked.pid'
> I0525 18:52:15.903939  1206 cni.cpp:888] Bind mounted '/proc/3985/ns/net' to 
> '/run/mesos/isolators/network/cni/c4b299e6-629a-4a99-bd88-cfbca0262b1a/ns' 
> for container c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:16.347486  1206 cni.cpp:1301] Got assigned IPv4 address 
> '192.168.110.2/24' from CNI network 'netcontiv' for container 
> c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:16.347533  1206 cni.cpp:1307] Got assigned IPv6 address '' from 
> CNI network 'netcontiv' for container c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:16.347687  1206 cni.cpp:1010] Unable to find DNS nameservers for 
> container c4b299e6-629a-4a99-bd88-cfbca0262b1a, using host '/etc/resolv.conf'
> I0525 18:52:24.579439  1206 containerizer.cpp:2508] Container 
> c4b299e6-629a-4a99-bd88-cfbca0262b1a has exited
> I0525 18:52:24.579493  1206 containerizer.cpp:2102] Destroying container 
> c4b299e6-629a-4a99-bd88-cfbca0262b1a in RUNNING state
> I0525 18:52:24.579560  1206 linux_launcher.cpp:505] Asked to destroy 
> container c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:24.580025  1206 linux_launcher.cpp:548] Using freezer to destroy 
> cgroup mesos/c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:24.580930  1206 cgroups.cpp:2692] Freezing cgroup 
> /sys/fs/cgroup/freezer/mesos/c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:24.582156  1206 cgroups.cpp:1405] Successfully froze cgroup 
> /sys/fs/cgroup/freezer/mesos/c4b299e6-629a-4a99-bd88-cfbca0262b1a after 
> 1.18784ms
> I0525 18:52:24.583359  1206 cgroups.cpp:2710] Thawing cgroup 
> /sys/fs/cgroup/freezer/mesos/c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:24.584491  1206 cgroups.cpp:1434] Successfully thawed cgroup 
> /sys/fs/cgroup/freezer/mesos/c4b299e6-629a-4a99-bd88-cfbca0262b1a after 
> 1.093888ms
> I0525 18:52:24.681495  1203 cni.cpp:1479] Unmounted the network namespace 
> handle 
> '/run/mesos/isolators/network/cni/c4b299e6-629a-4a99-bd88-cfbca0262b1a/ns' 
> for container c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:24.681591  1203 cni.cpp:1490] Removed the container directory 
> '/run/mesos/isolators/network/cni/c4b299e6-629a-4a99-bd88-cfbca0262b1a'
> I0525 18:52:24.691004  1203 slave.cpp:5168] Executor 
> 'container.1467.373c1d9b-4138-11e7-9117-024221dd5669' of framework 
> 00e6894c-d896-4a3d-8e79-679077f2af81- terminated with signal Killed
> I0525 18:52:24.691063  1203 slave.cpp:4215] Handling status update 
> TASK_FAILED (UUID: e90f3161-d136-4607-a67c-a621df9e82e4) for task 
> container.1467.373c1d9b-4138-11e7-9117-024221dd5669 of framework 
> 00e6894c-d896-4a3d-8e79-679077f2af81- from @0.0.0.0:0
> Log from sandbox:
> I0525 18:52:36.583499  4041 exec.cpp:162] Version: 1.3.0
> E0525 18:52:39.593489  4050 process.cpp:2450] Failed to shutdown socket with 
> fd 6, address 192.168.110.2:34176: Transport endpoint is not connected
> I0525 18:52:39.593582  4048 exec.cpp:497] Agent exited ... shutting down
> However, when deploying a task without ipAddress field, mesos slave launches 
> a task successfully.
> Tested with various Mesos/Marathon/Contiv versions.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7565) Container with "Contiv" networking fails upon startup

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-7565:
-
Affects Version/s: (was: 1.2.1)
   (was: 1.3.0)
   (was: 1.2.0)
 Priority: Major  (was: Critical)
  Component/s: (was: containerization)
  Summary: Container with "Contiv" networking fails upon startup  
(was: process.cpp:2450] Failed to shutdown socket with fd 6, address 
192.168.110.2:34176: Transport endpoint is not connected)

> Container with "Contiv" networking fails upon startup
> -
>
> Key: MESOS-7565
> URL: https://issues.apache.org/jira/browse/MESOS-7565
> Project: Mesos
>  Issue Type: Bug
>  Components: network
>Affects Versions: 1.2.2, 1.3.1
> Environment: centos 7.3
>Reporter: Hao Yixin
>
> When launching a task through Marathon and asking the task to assign an IP 
> (using Contiv networking):
> Log from mesos-slave:
> I0525 18:52:15.898908  1210 linux_launcher.cpp:429] Launching container 
> c4b299e6-629a-4a99-bd88-cfbca0262b1a and cloning with namespaces CLONE_NEWNS 
> | CLONE_NEWUTS | CLONE_NEWNET
> I0525 18:52:15.900668  1210 systemd.cpp:96] Assigned child process '3985' to 
> 'mesos_executors.slice'
> I0525 18:52:15.902612  1206 containerizer.cpp:1623] Checkpointing container's 
> forked pid 3985 to 
> '/var/lib/mesos/meta/slaves/00e6894c-d896-4a3d-8e79-679077f2af81-S4/frameworks/00e6894c-d896-4a3d-8e79-679077f2af81-/executors/container.1467.373c1d9b-4138-11e7-9117-024221dd5669/runs/c4b299e6-629a-4a99-bd88-cfbca0262b1a/pids/forked.pid'
> I0525 18:52:15.903939  1206 cni.cpp:888] Bind mounted '/proc/3985/ns/net' to 
> '/run/mesos/isolators/network/cni/c4b299e6-629a-4a99-bd88-cfbca0262b1a/ns' 
> for container c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:16.347486  1206 cni.cpp:1301] Got assigned IPv4 address 
> '192.168.110.2/24' from CNI network 'netcontiv' for container 
> c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:16.347533  1206 cni.cpp:1307] Got assigned IPv6 address '' from 
> CNI network 'netcontiv' for container c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:16.347687  1206 cni.cpp:1010] Unable to find DNS nameservers for 
> container c4b299e6-629a-4a99-bd88-cfbca0262b1a, using host '/etc/resolv.conf'
> I0525 18:52:24.579439  1206 containerizer.cpp:2508] Container 
> c4b299e6-629a-4a99-bd88-cfbca0262b1a has exited
> I0525 18:52:24.579493  1206 containerizer.cpp:2102] Destroying container 
> c4b299e6-629a-4a99-bd88-cfbca0262b1a in RUNNING state
> I0525 18:52:24.579560  1206 linux_launcher.cpp:505] Asked to destroy 
> container c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:24.580025  1206 linux_launcher.cpp:548] Using freezer to destroy 
> cgroup mesos/c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:24.580930  1206 cgroups.cpp:2692] Freezing cgroup 
> /sys/fs/cgroup/freezer/mesos/c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:24.582156  1206 cgroups.cpp:1405] Successfully froze cgroup 
> /sys/fs/cgroup/freezer/mesos/c4b299e6-629a-4a99-bd88-cfbca0262b1a after 
> 1.18784ms
> I0525 18:52:24.583359  1206 cgroups.cpp:2710] Thawing cgroup 
> /sys/fs/cgroup/freezer/mesos/c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:24.584491  1206 cgroups.cpp:1434] Successfully thawed cgroup 
> /sys/fs/cgroup/freezer/mesos/c4b299e6-629a-4a99-bd88-cfbca0262b1a after 
> 1.093888ms
> I0525 18:52:24.681495  1203 cni.cpp:1479] Unmounted the network namespace 
> handle 
> '/run/mesos/isolators/network/cni/c4b299e6-629a-4a99-bd88-cfbca0262b1a/ns' 
> for container c4b299e6-629a-4a99-bd88-cfbca0262b1a
> I0525 18:52:24.681591  1203 cni.cpp:1490] Removed the container directory 
> '/run/mesos/isolators/network/cni/c4b299e6-629a-4a99-bd88-cfbca0262b1a'
> I0525 18:52:24.691004  1203 slave.cpp:5168] Executor 
> 'container.1467.373c1d9b-4138-11e7-9117-024221dd5669' of framework 
> 00e6894c-d896-4a3d-8e79-679077f2af81- terminated with signal Killed
> I0525 18:52:24.691063  1203 slave.cpp:4215] Handling status update 
> TASK_FAILED (UUID: e90f3161-d136-4607-a67c-a621df9e82e4) for task 
> container.1467.373c1d9b-4138-11e7-9117-024221dd5669 of framework 
> 00e6894c-d896-4a3d-8e79-679077f2af81- from @0.0.0.0:0
> Log from sandbox:
> I0525 18:52:36.583499  4041 exec.cpp:162] Version: 1.3.0
> E0525 18:52:39.593489  4050 process.cpp:2450] Failed to shutdown socket with 
> fd 6, address 192.168.110.2:34176: Transport endpoint is not connected
> I0525 18:52:39.593582  4048 exec.cpp:497] Agent exited ... shutting down
> However, when deploying a task without ipAddress field, mesos slave launches 
> a task successfully.
> Tested with various Mesos/Marathon/Contiv versions.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7541) Cannot compile without pre-compiled headers on Windows

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-7541:
-
Description: 
Looks like we messed up an include at some point:

{noformat}
"C:\Users\andschwa\src\mesos\build\src\tests\mesos-tests.vcxproj" (default 
target) (1) ->
"C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj" (default target) 
(4) ->
(ClCompile target) ->
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(104): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(138): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(151): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(151): error 
C2131: expression did not evaluate to a constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(151): error 
C2051: case expression not constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(164): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(164): error 
C2131: expression did not evaluate to a constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(164): error 
C2051: case expression not constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(233): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(233): error 
C2131: expression did not evaluate to a constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(234): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(246): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(246): error 
C2512: 'Try': no appropriate default constructor available 
(compiling source file C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(233): error 
C2051: case expression not constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(293): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(293): error 
C2131: expression did not evaluate to a constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(293): error 
C2051: case expression not constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(322): error 
C2065: 'AF_INET6': 

[jira] [Updated] (MESOS-7541) Cannot compile without pre-compiled headers on Windows

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-7541:
-
Description: 
Looks like we messed up an include at some point:

{noformat}
"C:\Users\andschwa\src\mesos\build\src\tests\mesos-tests.vcxproj" (default 
target) (1) ->
"C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj" (default target) 
(4) ->
(ClCompile target) ->
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(104): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(138): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(151): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(151): error 
C2131: expression did not evaluate to a constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(151): error 
C2051: case expression not constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(164): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(164): error 
C2131: expression did not evaluate to a constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(164): error 
C2051: case expression not constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(233): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(233): error 
C2131: expression did not evaluate to a constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(234): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(246): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(246): error 
C2512: 'Try': no appropriate default constructor available 
(compiling source file C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mes
os-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(233): error 
C2051: case expression not constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(293): error 
C2065: 'AF_INET6': undeclared identifier (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(293): error 
C2131: expression did not evaluate to a constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(293): error 
C2051: case expression not constant (compiling source file 
C:\Users\andschwa\src\mesos\src\zookeeper\group.cpp) 
[C:\Users\andschwa\src\mesos\build\src\mesos-1.4.0.vcxproj]
  C:\Users\andschwa\src\mesos\3rdparty\stout\include\stout/ip.hpp(322): error 
C2065: 'AF_INET6': 

[jira] [Commented] (MESOS-7343) Add a ReviewBot for testing patches on Windows

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016360#comment-16016360
 ] 

Joseph Wu commented on MESOS-7343:
--

{code}
commit 7acec00b6aefa43b89bee3fa66579b2eded56af4
Author: Andrew Schwartzmeyer 
Date:   Thu May 18 12:56:42 2017 -0700

Windows: Fixed apply-reviews.py to retain line feeds.

Write using 'wb' instead of 'w' for binary mode. This writes the
downloaded patch file exactly as it came, instead of treating as text
and changing line endings. This resolves a bug where `git apply` doesn't
always work with CRLF endings in patch files.

Review: https://reviews.apache.org/r/59293/
{code}

> Add a ReviewBot for testing patches on Windows
> --
>
> Key: MESOS-7343
> URL: https://issues.apache.org/jira/browse/MESOS-7343
> Project: Mesos
>  Issue Type: Improvement
>  Components: reviewbot
> Environment: Windows 10
>Reporter: Andrew Schwartzmeyer
>Assignee: Andrew Schwartzmeyer
>  Labels: microsoft, windows
> Fix For: 1.4.0
>
>
> A major hindrance to development of Mesos for Windows is that, since the 
> platform support is so new, no tests their patches on Windows. Frequently 
> this causes upstream commits to master to break the Windows build (a recent 
> notable occasion was commit 5f159cdcb), which interrupts Windows developers 
> (read: me) in order to fix the build. Much of the problem is due to broken 
> processes failing to catch human error.
> One process we can immediately fix is the ReviewBot, which tests uploaded 
> patches for Ubuntu. Adding the same functionality but for Windows will go a 
> long way toward preventing upstream build breaks.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MESOS-7497) Remove CMake anti-pattern of `set(x "${x} ..")`.

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010992#comment-16010992
 ] 

Joseph Wu commented on MESOS-7497:
--

Yup, need to partially revert until we decide to up the minimum CMake version:
{code}
commit 47562d925a715eb174ad9738d6d188dd6bfe
Author: Joseph Wu 
Date:   Mon May 15 10:50:43 2017 -0700

CMake: Reverts usage of `string(APPEND ...)`.

This partially reverts commit 4ce689c8e2be386d0086acce63ce5b8bbab6c34b.

`string(APPEND ...)` was added in CMake 3.4, but the earliest required
version of CMake is 2.8.12.  `list(APPEND ...)` exists in 2.8.12,
so that pattern will not be reverted.
{code}

> Remove CMake anti-pattern of `set(x "${x} ..")`.
> 
>
> Key: MESOS-7497
> URL: https://issues.apache.org/jira/browse/MESOS-7497
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Andrew Schwartzmeyer
>Assignee: Andrew Schwartzmeyer
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MESOS-7497) Remove CMake anti-pattern of `set(x "${x} ..")`.

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010958#comment-16010958
 ] 

Joseph Wu commented on MESOS-7497:
--

{code}commit 4ce689c8e2be386d0086acce63ce5b8bbab6c34b
Author: Andrew Schwartzmeyer 
Date:   Wed May 10 14:08:06 2017 -0700

CMake: Use `list(APPEND x ...)` over `set(x ${x} ...)`.

This removes a CMake anti-pattern where `set` was used to append to a
variable, instead of the more explicit `list(APPEND)` or
`string(APPEND)` alternatives.

Review: https://reviews.apache.org/r/59156/
{code}

> Remove CMake anti-pattern of `set(x "${x} ..")`.
> 
>
> Key: MESOS-7497
> URL: https://issues.apache.org/jira/browse/MESOS-7497
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Andrew Schwartzmeyer
>Assignee: Andrew Schwartzmeyer
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7343) Add a ReviewBot for testing patches on Windows

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-7343:
-
Fix Version/s: (was: 1.3.0)
   1.4.0

> Add a ReviewBot for testing patches on Windows
> --
>
> Key: MESOS-7343
> URL: https://issues.apache.org/jira/browse/MESOS-7343
> Project: Mesos
>  Issue Type: Improvement
>  Components: reviewbot
> Environment: Windows 10
>Reporter: Andrew Schwartzmeyer
>Assignee: Andrew Schwartzmeyer
>  Labels: microsoft, windows
> Fix For: 1.4.0
>
>
> A major hindrance to development of Mesos for Windows is that, since the 
> platform support is so new, no tests their patches on Windows. Frequently 
> this causes upstream commits to master to break the Windows build (a recent 
> notable occasion was commit 5f159cdcb), which interrupts Windows developers 
> (read: me) in order to fix the build. Much of the problem is due to broken 
> processes failing to catch human error.
> One process we can immediately fix is the ReviewBot, which tests uploaded 
> patches for Ubuntu. Adding the same functionality but for Windows will go a 
> long way toward preventing upstream build breaks.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)