Re: Review Request 57215: Startup Annotation Scanning Takes Too Long

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57215/
---

(Updated March 2, 2017, 10:48 a.m.)


Review request for Ambari, Nate Cole and Robert Levas.


Bugs: AMBARI-20262
https://issues.apache.org/jira/browse/AMBARI-20262


Repository: ambari


Description
---

Currently, Ambari uses Spring's {{ClassPathScanningCandidateComponentProvider}} 
along with an {{AnnotationTypeFilter}} to find classes annotated in our 
classpath:

```
  ClassPathScanningCandidateComponentProvider scanner =
  new ClassPathScanningCandidateComponentProvider(false);

  // match only singletons that are eager listeners
  for (Class cls : classes) {
scanner.addIncludeFilter(new AnnotationTypeFilter(cls));
  }

  beanDefinitions = scanner.findCandidateComponents(AMBARI_PACKAGE);
```

This takes roughly 19 seconds on normal deployments. We can reduce this time to 
roughly 8 seconds by switching over to Google's {{ClassPath}} via our internal 
{{ClasspathScannerUtils}}


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerModule.java
 d8b8466 
  
ambari-server/src/main/java/org/apache/ambari/server/cleanup/ClasspathScannerUtils.java
 01487f0 
  
ambari-server/src/main/java/org/apache/ambari/server/cleanup/CleanupModule.java 
d2a7583 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
 8646e51 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
 40c7e50 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
 4d609b4 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/WidgetResourceProviderTest.java
 a4f6469 
  
ambari-server/src/test/java/org/apache/ambari/server/orm/InMemoryDefaultTestModule.java
 3090108 


Diff: https://reviews.apache.org/r/57215/diff/2/

Changes: https://reviews.apache.org/r/57215/diff/1-2/


Testing (updated)
---

mvn clean test


Thanks,

Jonathan Hurley

Re: Review Request 57215: Startup Annotation Scanning Takes Too Long

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57215/#review167677
---


Ship it!




Ship It!

- Nate Cole


On March 1, 2017, 3:51 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57215/
> ---
> 
> (Updated March 1, 2017, 3:51 p.m.)
> 
> 
> Review request for Ambari, Nate Cole and Robert Levas.
> 
> 
> Bugs: AMBARI-20262
> https://issues.apache.org/jira/browse/AMBARI-20262
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently, Ambari uses Spring's 
> {{ClassPathScanningCandidateComponentProvider}} along with an 
> {{AnnotationTypeFilter}} to find classes annotated in our classpath:
> 
> ```
>   ClassPathScanningCandidateComponentProvider scanner =
>   new ClassPathScanningCandidateComponentProvider(false);
> 
>   // match only singletons that are eager listeners
>   for (Class cls : classes) {
> scanner.addIncludeFilter(new AnnotationTypeFilter(cls));
>   }
> 
>   beanDefinitions = scanner.findCandidateComponents(AMBARI_PACKAGE);
> ```
> 
> This takes roughly 19 seconds on normal deployments. We can reduce this time 
> to roughly 8 seconds by switching over to Google's {{ClassPath}} via our 
> internal {{ClasspathScannerUtils}}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerModule.java
>  d8b8466 
>   
> ambari-server/src/main/java/org/apache/ambari/server/cleanup/ClasspathScannerUtils.java
>  01487f0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/cleanup/CleanupModule.java
>  d2a7583 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  8646e51 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/InMemoryDefaultTestModule.java
>  3090108 
> 
> 
> Diff: https://reviews.apache.org/r/57215/diff/1/
> 
> 
> Testing
> ---
> 
> PENDING
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 57215: Startup Annotation Scanning Takes Too Long

[Jonathan Hurley]

> On March 1, 2017, 4:52 p.m., Robert Nettleton wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
> > Line 695 (original), 694 (patched)
> > 
> >
> > Minor question:  Should these log messages really be at the INFO level? 
> >  
> > 
> > It seems like we already have too much noise in the ambari-server 
> > startup log, so it's not clear to me why these would be moved from DEBUG.

I debated this as well. I think it's nice to know that they were picked up, but 
then again, we probably don't need them at info ... I'll move these to `debug`


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57215/#review167580
---


On March 1, 2017, 3:51 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57215/
> ---
> 
> (Updated March 1, 2017, 3:51 p.m.)
> 
> 
> Review request for Ambari, Nate Cole and Robert Levas.
> 
> 
> Bugs: AMBARI-20262
> https://issues.apache.org/jira/browse/AMBARI-20262
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently, Ambari uses Spring's 
> {{ClassPathScanningCandidateComponentProvider}} along with an 
> {{AnnotationTypeFilter}} to find classes annotated in our classpath:
> 
> ```
>   ClassPathScanningCandidateComponentProvider scanner =
>   new ClassPathScanningCandidateComponentProvider(false);
> 
>   // match only singletons that are eager listeners
>   for (Class cls : classes) {
> scanner.addIncludeFilter(new AnnotationTypeFilter(cls));
>   }
> 
>   beanDefinitions = scanner.findCandidateComponents(AMBARI_PACKAGE);
> ```
> 
> This takes roughly 19 seconds on normal deployments. We can reduce this time 
> to roughly 8 seconds by switching over to Google's {{ClassPath}} via our 
> internal {{ClasspathScannerUtils}}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerModule.java
>  d8b8466 
>   
> ambari-server/src/main/java/org/apache/ambari/server/cleanup/ClasspathScannerUtils.java
>  01487f0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/cleanup/CleanupModule.java
>  d2a7583 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  8646e51 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/InMemoryDefaultTestModule.java
>  3090108 
> 
> 
> Diff: https://reviews.apache.org/r/57215/diff/1/
> 
> 
> Testing
> ---
> 
> PENDING
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 57215: Startup Annotation Scanning Takes Too Long

[Robert Nettleton]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57215/#review167580
---


Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
Line 695 (original), 694 (patched)


Minor question:  Should these log messages really be at the INFO level?  

It seems like we already have too much noise in the ambari-server startup 
log, so it's not clear to me why these would be moved from DEBUG.


- Robert Nettleton


On March 1, 2017, 8:51 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57215/
> ---
> 
> (Updated March 1, 2017, 8:51 p.m.)
> 
> 
> Review request for Ambari, Nate Cole and Robert Levas.
> 
> 
> Bugs: AMBARI-20262
> https://issues.apache.org/jira/browse/AMBARI-20262
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently, Ambari uses Spring's 
> {{ClassPathScanningCandidateComponentProvider}} along with an 
> {{AnnotationTypeFilter}} to find classes annotated in our classpath:
> 
> ```
>   ClassPathScanningCandidateComponentProvider scanner =
>   new ClassPathScanningCandidateComponentProvider(false);
> 
>   // match only singletons that are eager listeners
>   for (Class cls : classes) {
> scanner.addIncludeFilter(new AnnotationTypeFilter(cls));
>   }
> 
>   beanDefinitions = scanner.findCandidateComponents(AMBARI_PACKAGE);
> ```
> 
> This takes roughly 19 seconds on normal deployments. We can reduce this time 
> to roughly 8 seconds by switching over to Google's {{ClassPath}} via our 
> internal {{ClasspathScannerUtils}}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerModule.java
>  d8b8466 
>   
> ambari-server/src/main/java/org/apache/ambari/server/cleanup/ClasspathScannerUtils.java
>  01487f0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/cleanup/CleanupModule.java
>  d2a7583 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  8646e51 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/InMemoryDefaultTestModule.java
>  3090108 
> 
> 
> Diff: https://reviews.apache.org/r/57215/diff/1/
> 
> 
> Testing
> ---
> 
> PENDING
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 57215: Startup Annotation Scanning Takes Too Long

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57215/#review167568
---




ambari-server/src/main/java/org/apache/ambari/server/cleanup/ClasspathScannerUtils.java
Line 99 (original), 100 (patched)


This was previously comparing `Annotation` to `Class` which didn't match.



ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
Lines 526-527 (original), 526-527 (patched)


This is really the only important change here - essentially Spring's 
annotation scanning was taking like 19 seconds. We have a classpath scanner 
that uses Google's Reflections library which is much faster.


- Jonathan Hurley


On March 1, 2017, 3:51 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57215/
> ---
> 
> (Updated March 1, 2017, 3:51 p.m.)
> 
> 
> Review request for Ambari, Nate Cole and Robert Levas.
> 
> 
> Bugs: AMBARI-20262
> https://issues.apache.org/jira/browse/AMBARI-20262
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently, Ambari uses Spring's 
> {{ClassPathScanningCandidateComponentProvider}} along with an 
> {{AnnotationTypeFilter}} to find classes annotated in our classpath:
> 
> ```
>   ClassPathScanningCandidateComponentProvider scanner =
>   new ClassPathScanningCandidateComponentProvider(false);
> 
>   // match only singletons that are eager listeners
>   for (Class cls : classes) {
> scanner.addIncludeFilter(new AnnotationTypeFilter(cls));
>   }
> 
>   beanDefinitions = scanner.findCandidateComponents(AMBARI_PACKAGE);
> ```
> 
> This takes roughly 19 seconds on normal deployments. We can reduce this time 
> to roughly 8 seconds by switching over to Google's {{ClassPath}} via our 
> internal {{ClasspathScannerUtils}}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerModule.java
>  d8b8466 
>   
> ambari-server/src/main/java/org/apache/ambari/server/cleanup/ClasspathScannerUtils.java
>  01487f0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/cleanup/CleanupModule.java
>  d2a7583 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  8646e51 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/InMemoryDefaultTestModule.java
>  3090108 
> 
> 
> Diff: https://reviews.apache.org/r/57215/diff/1/
> 
> 
> Testing
> ---
> 
> PENDING
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Review Request 57215: Startup Annotation Scanning Takes Too Long

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57215/
---

Review request for Ambari, Nate Cole and Robert Levas.


Bugs: AMBARI-20262
https://issues.apache.org/jira/browse/AMBARI-20262


Repository: ambari


Description
---

Currently, Ambari uses Spring's {{ClassPathScanningCandidateComponentProvider}} 
along with an {{AnnotationTypeFilter}} to find classes annotated in our 
classpath:

```
  ClassPathScanningCandidateComponentProvider scanner =
  new ClassPathScanningCandidateComponentProvider(false);

  // match only singletons that are eager listeners
  for (Class cls : classes) {
scanner.addIncludeFilter(new AnnotationTypeFilter(cls));
  }

  beanDefinitions = scanner.findCandidateComponents(AMBARI_PACKAGE);
```

This takes roughly 19 seconds on normal deployments. We can reduce this time to 
roughly 8 seconds by switching over to Google's {{ClassPath}} via our internal 
{{ClasspathScannerUtils}}


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerModule.java
 d8b8466 
  
ambari-server/src/main/java/org/apache/ambari/server/cleanup/ClasspathScannerUtils.java
 01487f0 
  
ambari-server/src/main/java/org/apache/ambari/server/cleanup/CleanupModule.java 
d2a7583 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
 8646e51 
  
ambari-server/src/test/java/org/apache/ambari/server/orm/InMemoryDefaultTestModule.java
 3090108 


Diff: https://reviews.apache.org/r/57215/diff/1/


Testing
---

PENDING


Thanks,

Jonathan Hurley