[GitHub] kpavel opened a new pull request #789: Add assets annotation tags when assets exported
kpavel opened a new pull request #789: Add assets annotation tags when assets exported URL: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/789 Currently exported assets manifest is missing annotations assets have in OpenWhisk. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr commented on issue #3419: make OW context information available to Action
csantanapr commented on issue #3419: make OW context information available to Action URL: https://github.com/apache/incubator-openwhisk/issues/3419#issuecomment-372110134 Hum that was on the init or run? They should be there on the run On the init not today, but we can think which one could be added to init. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ScottChapman opened a new issue #101: please add ignore_certs env var
ScottChapman opened a new issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101 When running test code for openwhisk actions it is often necessary to ignore certificates since it is running openwhisk in travis. It looks like the other constructor parameters are already handled this way. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #101: please add ignore_certs env var
rabbah commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372122094 You can pass a parameter: ignore_certs. Turns off server SSL/TLS certificate verification. This allows the client to be used against local deployments of OpenWhisk with a self-signed certificate. Defaults to false. If you?re running locally you can also set NODE_TLS_REJECT_UNAUTHORIZED This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ScottChapman commented on issue #101: please add ignore_certs env var
ScottChapman commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372123671 Yes, I am currently doing that in order to run my test code. In my particular case I have an action that is using this module to be able to invoke other actions. I don't want to have to change my production action code to always ignore certificates. If this env var was available then I could include the variable in my travis config, and when tests are run certs would be ignored. But they would be enforced when running in production on IBM Cloud This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr commented on issue #101: please add ignore_certs env var
csantanapr commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372123864 We could improve the library by detecting if apihost its an IP address then ignore ssl if it?s a hostname then do not ignore. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #101: please add ignore_certs env var
rabbah commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372125100 It?s a heuristic that would be prone to error. I use an alias ?openwhisk? as my api host for my local deployment. I?d rather have an explicit toggle (param, env var), or use the http port instead (10001). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] starpit commented on issue #101: please add ignore_certs env var
starpit commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372125490 i think scott is simply suggesting that the openwhisk npm be updated to accept an env var for ignore_certs. it accepts env vars for everything else, already: https://github.com/apache/incubator-openwhisk-client-js/blob/master/lib/client.js#L64-L72 note the conspicuous anomaly of `ignore_certs` i'd like to go further, and have the npm parse out `process.env.WSK_PROPS_FILE || ~/.wskprops` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #101: please add ignore_certs env var
rabbah commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372125855 Why not just set NODE_TLS_REJECT_UNAUTHORIZED This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] starpit commented on issue #101: please add ignore_certs env var
starpit commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372125938 Maybe users want to be more selective. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #101: please add ignore_certs env var
rabbah commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372125994 The wsk prop file doesn?t help. We rejected the pr which records the certificate validity. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #101: please add ignore_certs env var
rabbah commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372126177 I?m just pointing out we have at least three ways to achieve the workaround today: parameter, node?s own env var, or use an http port. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] starpit commented on issue #101: please add ignore_certs env var
starpit commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372126570 i do see what scott is after. he doesn't want to have to modify his code for various setups. i certainly wouldn't feel comfortable disabling cert checking globally (for all sites). he could introduce the env var support in *his* code (`ignore_certs: process.env.IGNORE_CERTS`), but why not just have this support in the npm directly? going further: openwhisk already passes __OW_API_HOST, etc. why not (also) inform the code that it (openwhisk) is using insecure certs, so that the code can configure itself accordingly? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] starpit commented on issue #101: please add ignore_certs env var
starpit commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372126570 i do see what scott is after. he doesn't want to have to modify his code for various setups. i certainly wouldn't feel comfortable disabling cert checking globally (for all sites). he could introduce the env var support in *his* code (`ignore_certs: process.env.IGNORE_CERTS`), but why not just have this support in the npm directly? going further: openwhisk already passes __OW_API_HOST, etc. why not (also) inform the code that it (openwhisk) is using insecure certs, so that the code can configure itself accordingly? still, your suggestion to use http:10001 seems the easiest for now. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr commented on issue #101: please add ignore_certs env var
csantanapr commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372126903 FWIW I had same problem on swift4 sdk testing and use an override baseURL and use controller host and port 10001 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr commented on issue #101: please add ignore_certs env var
csantanapr commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372126903 FWIW I had same problem on swift4 sdk testing and use an override baseURL action Param and use controller host and port 10001 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #101: please add ignore_certs env var
rabbah commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372128750 Scott can you clarify how you?re developing and running? Are you writing js/node code and running it with node against a local openwhisk deployment, or are you creating actions and running them using a local openwhisk deployment? If it?s the former, your best option is to use the node tls setting or change the api host to use http://...:10001. If it?s the latter you will need to pass the ignore cert parameter or again configure the deployment to use the http port 10001. Anything more if you?re running against a local openwhisk deployment requires backend support and changes in more than just the node runtime to support a specific toggle that disables the tls check for openwhisk. I would rather spend the effort and automate a signed certificate or enable port 80 in the edge host and eschew the problem entirely. The rest are all hacks, some easier that others to realize. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr commented on issue #101: please add ignore_certs env var
csantanapr commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372129178 >I would rather spend the effort and automate a signed certificate or enable port 80 in the edge host and eschew the problem entirely. The rest are all hacks, some easier that others to realize. I really like this better to deploy in Travis and configure ssl cert and local hostname But this brings two difficult problems to solve how to configure the dns for the action runtime and how to insert the certs to trust the certificate. I think is doable by pulling the runtime and then modifying the image or rebuildin with the dns and certs inside the image. Enabling port 80 and using http on the edge can be quick solution also. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dgrove-oss commented on issue #3411: refactor KubernetesClient to separate invokerAgent; add forwarding LogStoreProvider
dgrove-oss commented on issue #3411: refactor KubernetesClient to separate invokerAgent; add forwarding LogStoreProvider URL: https://github.com/apache/incubator-openwhisk/pull/3411#issuecomment-372129332 PG2 / 2909 ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr commented on issue #101: please add ignore_certs env var
csantanapr commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372129389 I just realized I?m an idiot I should have done this for swift4 ssl sdk problem This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr opened a new issue #61: tag repos with version after first release with openwhisk@1.0.0
csantanapr opened a new issue #61: tag repos with version after first release with openwhisk@1.0.0 URL: https://github.com/apache/incubator-openwhisk-release/issues/61 After is declared done, all the repos included in the release at the respective git hash should be tag with a single label that is easy to distinguish the marker in the individual repos. I propose to uste convention `openwhisk@` For the first release this will be in practice `openwhisk@1.0.0` and son on `openwhisk@1.0.1`, `openwhisk@1.0.2`, `openwhisk@1.0.3`etc.. We can close this issue after release and repos tagged. cc @houshengbo This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #101: please add ignore_certs env var
rabbah commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372128750 Scott can you clarify how you?re developing and running? Are you writing js/node code and running it with node against a local openwhisk deployment, or are you creating actions and running them using a local openwhisk deployment? If it?s the former, your best option is to use the node tls setting or change the api host to use http://...:10001. If it?s the latter you will need to pass the ignore cert parameter or again configure the deployment to use the http port 10001. Anything more if you?re running against a local openwhisk deployment requires backend support and changes in more than just the node runtime to support a specific toggle that disables the tls check for openwhisk. I would rather spend the effort and automate a signed certificate or enable port 80 in the edge host and eschew the problem entirely. The rest are all hacks, some easier than others to realize. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ScottChapman commented on issue #101: please add ignore_certs env var
ScottChapman commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372140430 @rabbah - I am building a "Template" for IBM Cloud Functions. Part of that is to implement automated tests that run in a travis build. Nearly all the actions in my Template leverage other actions in the same package (using this node module). I can imagine it would be a useful feature during dev/test to be able to run against essentially an insecure openwhisk deployment. My request is simply to make it consistent with the other constructor parameters that can be set via env vars. I prefer explicit behavior over implicit as it is less likely to violate the rule of least astonishment. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #101: please add ignore_certs env var
rabbah commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372140846 Thanks for the clarification. So these are actions. The constructors accept the namespace and api host optionally from the environment. These are provided by the backend. So we would need back end support, part of a deployment, to enable a new environment variable to toggle the TLS verification. The way I?ve done this in my own code is to set the ignore cert parameter in the constructor using args.ignore_cert || false. You can modify Travis to then pass this flag through in the tests. To enable this in the backend we?d need a deployment parameter, and modifications in the invoker which generates the environment for the action. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ScottChapman commented on issue #101: please add ignore_certs env var
ScottChapman commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372141502 Yes, I've similarly implemented a work-around by altering my code, and modified my travis. I'm doing it as a package param, but still my code needs to leverage it. I do understand that there would need to be some backend support. Sort of surprised it doesn't exist already so that actions running in a dev OW deployment can leverage actions in the same environment in a similar fashion. But yea, I realize now it is not as simple as supporting the env var in the node code... But I can imagine this being a common use-case (dev/test against insecure OW deployment) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jasonpet opened a new pull request #139: fix failing test and refactor others into appropriate suites
jasonpet opened a new pull request #139: fix failing test and refactor others into appropriate suites URL: https://github.com/apache/incubator-openwhisk-package-alarms/pull/139 fix for the AlarmsMultiWorkerTests to use whisk properties instead of pure config simply moved some of the other tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jasonpet commented on issue #139: fix failing test and refactor others into appropriate suites
jasonpet commented on issue #139: fix failing test and refactor others into appropriate suites URL: https://github.com/apache/incubator-openwhisk-package-alarms/pull/139#issuecomment-372160930 @dubeejw This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dubeejw closed pull request #139: fix failing test and refactor others into appropriate suites
dubeejw closed pull request #139: fix failing test and refactor others into
appropriate suites
URL: https://github.com/apache/incubator-openwhisk-package-alarms/pull/139
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/tests/src/test/scala/system/health/AlarmsHealthFeedTests.scala
b/tests/src/test/scala/system/health/AlarmsHealthFeedTests.scala
index 90ae224..3db4b0b 100644
--- a/tests/src/test/scala/system/health/AlarmsHealthFeedTests.scala
+++ b/tests/src/test/scala/system/health/AlarmsHealthFeedTests.scala
@@ -18,10 +18,10 @@ package system.health
import common._
import org.junit.runner.RunWith
+import org.scalatest.FlatSpec
import org.scalatest.junit.JUnitRunner
-import org.scalatest.{FlatSpec, Inside}
-import spray.json.DefaultJsonProtocol.{BooleanJsonFormat, IntJsonFormat,
LongJsonFormat, StringJsonFormat}
-import spray.json.{JsObject, JsString, pimpAny}
+import spray.json.DefaultJsonProtocol.{IntJsonFormat, LongJsonFormat,
StringJsonFormat}
+import spray.json.pimpAny
/**
* Tests for alarms trigger service
@@ -30,7 +30,6 @@ import spray.json.{JsObject, JsString, pimpAny}
class AlarmsHealthFeedTests
extends FlatSpec
with TestHelpers
-with Inside
with WskTestHelpers {
val wskprops = WskProps()
@@ -196,274 +195,4 @@ class AlarmsHealthFeedTests
activationsAfterInterval should be(2)
}
-it should "update cron, startDate and stopDate parameters" in
withAssetCleaner(wskprops) {
-(wp, assetHelper) =>
-implicit val wskProps = wp
-val triggerName = s"dummyAlarmsTrigger-${System.currentTimeMillis}"
-val packageName = "dummyAlarmsPackage"
-
-// the package alarms should be there
-val packageGetResult = wsk.pkg.get("/whisk.system/alarms")
-println("fetched package alarms")
-packageGetResult.stdout should include("ok")
-
-// create package binding
-assetHelper.withCleaner(wsk.pkg, packageName) {
-(pkg, name) => pkg.bind("/whisk.system/alarms", name)
-}
-
-val cron = "* * * * *"
-val startDate = System.currentTimeMillis + (1000 * 30)
-val stopDate = startDate + (1000 * 100)
-
-// create trigger feed
-println(s"Creating trigger: $triggerName")
-assetHelper.withCleaner(wsk.trigger, triggerName) {
-(trigger, name) =>
-trigger.create(name, feed = Some(s"$packageName/alarm"),
parameters = Map(
-"cron" -> cron.toJson,
-"startDate" -> startDate.toJson,
-"stopDate" -> stopDate.toJson))
-}
-
-
-val actionName = s"$packageName/alarm"
-val readRunResult = wsk.action.invoke(actionName, parameters = Map(
-"triggerName" -> triggerName.toJson,
-"lifecycleEvent" -> "READ".toJson,
-"authKey" -> wskProps.authKey.toJson
-))
-
-withActivation(wsk.activation, readRunResult) {
-activation => activation.response.success shouldBe true
-
-inside(activation.response.result) {
-case Some(result) =>
-val config =
result.getFields("config").head.asInstanceOf[JsObject].fields
-val status =
result.getFields("status").head.asInstanceOf[JsObject].fields
-
-config should contain("cron" -> cron.toJson)
-config should contain("startDate" ->
startDate.toJson)
-config should contain("stopDate" ->
stopDate.toJson)
-
-status should contain("active" -> true.toJson)
-status should contain key "dateChanged"
-status should contain key "dateChangedISO"
-status should not(contain key "reason")
-}
-}
-
-val updatedCron = "*/2 * * * *"
-val updatedStartDate = System.currentTimeMillis + (1000 * 30)
-val updatedStopDate = updatedStartDate + (1000 * 100)
-
-val updateRunAction = wsk.action.invoke(actionName, parameters =
Map(
-"triggerName" -> triggerName.toJson,
-"lifecycleEvent" -> "UPDATE".toJson,
-"authKey" -> wskProps.authKey.toJson,
-"cron" -> updatedCron.toJson,
-"startDate" -> updatedStartDate.toJson,
-"stopDate" -> updatedStopDate.toJson
-))
-
-withActivation(wsk.activation, updateRunAction) {
-activation =>
[GitHub] jasonpet opened a new pull request #160: fix failing test and other test cleanup
jasonpet opened a new pull request #160: fix failing test and other test cleanup URL: https://github.com/apache/incubator-openwhisk-package-cloudant/pull/160 fix for the CloudantMultiWorkerTests to use whisk properties instead of pureconfig test cleanup This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jasonpet commented on issue #160: fix failing test and other test cleanup
jasonpet commented on issue #160: fix failing test and other test cleanup URL: https://github.com/apache/incubator-openwhisk-package-cloudant/pull/160#issuecomment-372166177 @dubeejw This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jiangpengcheng commented on issue #3232: Making prewarm kind (and count) configurable
jiangpengcheng commented on issue #3232: Making prewarm kind (and count) configurable URL: https://github.com/apache/incubator-openwhisk/pull/3232#issuecomment-372166561 is this pr alive? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #3232: Making prewarm kind (and count) configurable
rabbah commented on issue #3232: Making prewarm kind (and count) configurable URL: https://github.com/apache/incubator-openwhisk/pull/3232#issuecomment-372168758 Yes. We?d like to get this in for sure. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dubeejw closed pull request #160: fix failing test and other test cleanup
dubeejw closed pull request #160: fix failing test and other test cleanup
URL: https://github.com/apache/incubator-openwhisk-package-cloudant/pull/160
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/tests/src/test/scala/system/health/CloudantHealthFeedTests.scala
b/tests/src/test/scala/system/health/CloudantHealthFeedTests.scala
index bb342d4..4c9e0b7 100644
--- a/tests/src/test/scala/system/health/CloudantHealthFeedTests.scala
+++ b/tests/src/test/scala/system/health/CloudantHealthFeedTests.scala
@@ -64,8 +64,8 @@ class CloudantHealthFeedTests
(wp, assetHelper) =>
implicit val wskprops = wp // shadow global props and make implicit
val triggerName =
s"dummyCloudantTrigger-${System.currentTimeMillis}"
-val ruleName = s"dummyAlarmsRule-${System.currentTimeMillis}"
-val actionName = s"dummyAlarmsAction-${System.currentTimeMillis}"
+val ruleName = s"dummyCloudantRule-${System.currentTimeMillis}"
+val actionName = s"dummyCloudantAction-${System.currentTimeMillis}"
val packageName = "dummyCloudantPackage"
val feed = "changes"
@@ -134,8 +134,8 @@ class CloudantHealthFeedTests
(wp, assetHelper) =>
implicit val wskprops = wp // shadow global props and make implicit
val triggerName =
s"dummyCloudantTrigger-${System.currentTimeMillis}"
-val ruleName = s"dummyAlarmsRule-${System.currentTimeMillis}"
-val actionName = s"dummyAlarmsAction-${System.currentTimeMillis}"
+val ruleName = s"dummyCloudantRule-${System.currentTimeMillis}"
+val actionName = s"dummyCloudantAction-${System.currentTimeMillis}"
val packageName = "dummyCloudantPackage"
val feed = "changes"
diff --git
a/tests/src/test/scala/system/packages/CloudantAccountActionsTests.scala
b/tests/src/test/scala/system/packages/CloudantAccountActionsTests.scala
index 3dee996..343de72 100644
--- a/tests/src/test/scala/system/packages/CloudantAccountActionsTests.scala
+++ b/tests/src/test/scala/system/packages/CloudantAccountActionsTests.scala
@@ -62,7 +62,7 @@ class CloudantAccountActionsTests extends FlatSpec
//create database
println("Invoking the create-database action.")
-withActivation(wsk.activation,
wsk.action.invoke(s"${packageName}/create-database",
+withActivation(wsk.activation,
wsk.action.invoke(s"$packageName/create-database",
Map("dbname" -> dbName.toJson))) {
activation =>
activation.response.success shouldBe true
@@ -95,7 +95,7 @@ class CloudantAccountActionsTests extends FlatSpec
//create database
println("Invoking the create-database action.")
-withActivation(wsk.activation,
wsk.action.invoke(s"${packageName}/create-database")) {
+withActivation(wsk.activation,
wsk.action.invoke(s"$packageName/create-database")) {
activation =>
activation.response.success shouldBe false
val result = activation.response.result.get
@@ -125,7 +125,7 @@ class CloudantAccountActionsTests extends FlatSpec
}
println("Invoking the read-database action.")
-withActivation(wsk.activation,
wsk.action.invoke(s"${packageName}/read-database",
+withActivation(wsk.activation,
wsk.action.invoke(s"$packageName/read-database",
Map("dbname" -> credential.dbname.toJson))) {
activation =>
activation.response.success shouldBe true
@@ -157,7 +157,7 @@ class CloudantAccountActionsTests extends FlatSpec
}
println("Invoking the read-database action.")
-withActivation(wsk.activation,
wsk.action.invoke(s"${packageName}/read-database",
+withActivation(wsk.activation,
wsk.action.invoke(s"$packageName/read-database",
Map("dbname" -> "doesNotExistDB".toJson))) {
activation =>
activation.response.success shouldBe false
@@ -188,14 +188,14 @@ class CloudantAccountActionsTests extends FlatSpec
}
println("Invoking the delete-database action.")
-withActivation(wsk.activation,
wsk.action.invoke(s"${packageName}/delete-database",
+withActivation(wsk.activation,
wsk.action.invoke(s"$packageName/delete-database",
Map("dbname" -> credential.dbname.toJson))) {
activation =>
activation.response.success shouldBe true
}
[GitHub] csantanapr commented on issue #101: please add ignore_certs env var
csantanapr commented on issue #101: please add ignore_certs env var URL: https://github.com/apache/incubator-openwhisk-client-js/issues/101#issuecomment-372188358 @rabbah Just to be sure we want to disable tls only for whisk api but we should not disable for other external url (ie watson url, weather api url, etc) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dubeejw opened a new pull request #3421: Add ElasticSearch Log Store
dubeejw opened a new pull request #3421: Add ElasticSearch Log Store URL: https://github.com/apache/incubator-openwhisk/pull/3421 ## Description ## Related issue and scope - [ ] I opened an issue to propose and discuss this change (#) ## My changes affect the following components - [ ] API - [ ] Controller - [ ] Message Bus (e.g., Kafka) - [ ] Loadbalancer - [ ] Invoker - [ ] Intrinsic actions (e.g., sequences, conductors) - [ ] Data stores (e.g., CouchDB) - [ ] Tests - [ ] Deployment - [ ] CLI - [ ] General tooling - [ ] Documentation ## Types of changes - [ ] Bug fix (generally a non-breaking change which closes an issue). - [ ] Enhancement or new feature (adds new functionality). - [ ] Breaking change (a bug fix or enhancement which changes existing behavior). ## Checklist: - [ ] I signed an [Apache CLA](https://github.com/apache/incubator-openwhisk/blob/master/CONTRIBUTING.md). - [ ] I reviewed the [style guides](https://github.com/apache/incubator-openwhisk/wiki/Contributing:-Git-guidelines#code-readiness) and followed the recommendations (Travis CI will check :). - [ ] I added tests to cover my changes. - [ ] My changes require further changes to the documentation. - [ ] I updated the documentation where necessary. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #3421: Add ElasticSearch Log Store
rabbah commented on issue #3421: Add ElasticSearch Log Store URL: https://github.com/apache/incubator-openwhisk/pull/3421#issuecomment-372201237 We should also provide (seperatly) an optional elastic playbook (related to #3390). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on a change in pull request #3421: Add ElasticSearch Log Store
rabbah commented on a change in pull request #3421: Add ElasticSearch Log Store
URL:
https://github.com/apache/incubator-openwhisk/pull/3421#discussion_r173699319
##
File path:
common/scala/src/main/scala/whisk/core/containerpool/logging/ElasticSearchLogStore.scala
##
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.logging
+
+import java.nio.file.{Path, Paths}
+import java.time.LocalDate
+import java.time.format.DateTimeFormatter
+
+import akka.actor.ActorSystem
+import akka.http.scaladsl.model.HttpRequest
+import akka.http.scaladsl.model.Uri
+import akka.stream.scaladsl.Flow
+import akka.http.scaladsl.model.HttpResponse
+import akka.http.scaladsl.model._
+
+import whisk.common.AkkaLogging
+import whisk.core.entity.{ActivationLogs, Identity, WhiskActivation}
+import whisk.core.containerpool.logging.ElasticSearchJsonProtocol._
+import whisk.core.ConfigKeys
+
+import scala.concurrent.Future
+import scala.concurrent.Promise
+import scala.util.Try
+
+import spray.json._
+
+import pureconfig._
+
+// Schema of logs in ES
+case class UserLogEntry(message: String, tenantId: String, stream: String,
time: String, action: String)
+
+object UserLogEntry extends DefaultJsonProtocol {
+ implicit val serdes =
+jsonFormat(UserLogEntry.apply, "message", "ALCH_TENANT_ID", "stream_str",
"time_date", "action_str")
Review comment:
hmm is the third field generic?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rabbah commented on a change in pull request #3421: Add ElasticSearch Log Store
rabbah commented on a change in pull request #3421: Add ElasticSearch Log Store
URL:
https://github.com/apache/incubator-openwhisk/pull/3421#discussion_r173699456
##
File path:
common/scala/src/main/scala/whisk/core/containerpool/logging/ElasticSearchLogStore.scala
##
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.logging
+
+import java.nio.file.{Path, Paths}
+import java.time.LocalDate
+import java.time.format.DateTimeFormatter
+
+import akka.actor.ActorSystem
+import akka.http.scaladsl.model.HttpRequest
+import akka.http.scaladsl.model.Uri
+import akka.stream.scaladsl.Flow
+import akka.http.scaladsl.model.HttpResponse
+import akka.http.scaladsl.model._
+
+import whisk.common.AkkaLogging
+import whisk.core.entity.{ActivationLogs, Identity, WhiskActivation}
+import whisk.core.containerpool.logging.ElasticSearchJsonProtocol._
+import whisk.core.ConfigKeys
+
+import scala.concurrent.Future
+import scala.concurrent.Promise
+import scala.util.Try
+
+import spray.json._
+
+import pureconfig._
+
+// Schema of logs in ES
+case class UserLogEntry(message: String, tenantId: String, stream: String,
time: String, action: String)
+
+object UserLogEntry extends DefaultJsonProtocol {
+ implicit val serdes =
+jsonFormat(UserLogEntry.apply, "message", "ALCH_TENANT_ID", "stream_str",
"time_date", "action_str")
+}
+
+case class ElasticSearchLogStoreConfig(protocol: String,
+ host: String,
+ port: Int,
+ logMessageField: String,
+ activationIdField: String,
+ requiredHeaders: String)
+
+/**
+ * Docker based implementation of a LogStore.
+ *
+ * Relies on docker's implementation details with regards to the JSON
log-driver. When using the JSON log-driver
+ * docker writes stdout/stderr to a JSON formatted file which is read by this
store. Logs are written in the
+ * activation record itself.
+ *
+ * Additionally writes logs to a separate file which can be processed by any
backend service asynchronously.
+ */
+class ElasticSearchLogStore(
+ system: ActorSystem,
+ httpFlow: Option[Flow[(HttpRequest, Promise[HttpResponse]),
(Try[HttpResponse], Promise[HttpResponse]), Any]] = None,
+ destinationDirectory: Path = Paths.get("logs"),
+ elasticSearchConfig: ElasticSearchLogStoreConfig =
+loadConfigOrThrow[ElasticSearchLogStoreConfig](ConfigKeys.elasticSearch))
+extends DockerToActivationFileLogStore(system, destinationDirectory) {
+
+ implicit val actorSystem = system
Review comment:
Which of these fields need to be public?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rabbah commented on a change in pull request #3421: Add ElasticSearch Log Store
rabbah commented on a change in pull request #3421: Add ElasticSearch Log Store
URL:
https://github.com/apache/incubator-openwhisk/pull/3421#discussion_r173699804
##
File path: common/scala/src/main/scala/whisk/http/PoolingRestClient.scala
##
@@ -55,16 +63,18 @@ class PoolingRestClient(protocol: String, host: String,
port: Int, queueSize: In
Http().cachedHostConnectionPoolHttps[Promise[HttpResponse]](host = host,
port = port)
}
+ private val defaultHttpFlow = pool.mapMaterializedValue { x =>
+poolPromise.success(x); x
+ }
+
private val poolPromise = Promise[HostConnectionPool]
// Additional queue in case all connections are busy. Should hardly ever be
// filled in practice but can be useful, e.g., in tests starting many
// asynchronous requests in a very short period of time.
private val requestQueue = Source
.queue(queueSize, OverflowStrategy.dropNew)
-.via(pool.mapMaterializedValue { x =>
- poolPromise.success(x); x
-})
+.via(httpFlow.getOrElse(defaultHttpFlow))
.toMat(Sink.foreach({
case ((Success(response), p)) => p.success(response)
Review comment:
I can?t tell just from looking at these few lines of the one promise is
being competed twice or if it?s two different promises.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] dubeejw commented on a change in pull request #3421: Add ElasticSearch Log Store
dubeejw commented on a change in pull request #3421: Add ElasticSearch Log Store
URL:
https://github.com/apache/incubator-openwhisk/pull/3421#discussion_r173700856
##
File path:
common/scala/src/main/scala/whisk/core/containerpool/logging/ElasticSearchLogStore.scala
##
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.logging
+
+import java.nio.file.{Path, Paths}
+import java.time.LocalDate
+import java.time.format.DateTimeFormatter
+
+import akka.actor.ActorSystem
+import akka.http.scaladsl.model.HttpRequest
+import akka.http.scaladsl.model.Uri
+import akka.stream.scaladsl.Flow
+import akka.http.scaladsl.model.HttpResponse
+import akka.http.scaladsl.model._
+
+import whisk.common.AkkaLogging
+import whisk.core.entity.{ActivationLogs, Identity, WhiskActivation}
+import whisk.core.containerpool.logging.ElasticSearchJsonProtocol._
+import whisk.core.ConfigKeys
+
+import scala.concurrent.Future
+import scala.concurrent.Promise
+import scala.util.Try
+
+import spray.json._
+
+import pureconfig._
+
+// Schema of logs in ES
+case class UserLogEntry(message: String, tenantId: String, stream: String,
time: String, action: String)
+
+object UserLogEntry extends DefaultJsonProtocol {
+ implicit val serdes =
+jsonFormat(UserLogEntry.apply, "message", "ALCH_TENANT_ID", "stream_str",
"time_date", "action_str")
Review comment:
Will need to make some of those fields configurable with pureconfig, I
believe.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] dubeejw commented on a change in pull request #3421: Add ElasticSearch Log Store
dubeejw commented on a change in pull request #3421: Add ElasticSearch Log Store
URL:
https://github.com/apache/incubator-openwhisk/pull/3421#discussion_r173701109
##
File path:
common/scala/src/main/scala/whisk/core/containerpool/logging/ElasticSearchLogStore.scala
##
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.logging
+
+import java.nio.file.{Path, Paths}
+import java.time.LocalDate
+import java.time.format.DateTimeFormatter
+
+import akka.actor.ActorSystem
+import akka.http.scaladsl.model.HttpRequest
+import akka.http.scaladsl.model.Uri
+import akka.stream.scaladsl.Flow
+import akka.http.scaladsl.model.HttpResponse
+import akka.http.scaladsl.model._
+
+import whisk.common.AkkaLogging
+import whisk.core.entity.{ActivationLogs, Identity, WhiskActivation}
+import whisk.core.containerpool.logging.ElasticSearchJsonProtocol._
+import whisk.core.ConfigKeys
+
+import scala.concurrent.Future
+import scala.concurrent.Promise
+import scala.util.Try
+
+import spray.json._
+
+import pureconfig._
+
+// Schema of logs in ES
+case class UserLogEntry(message: String, tenantId: String, stream: String,
time: String, action: String)
+
+object UserLogEntry extends DefaultJsonProtocol {
+ implicit val serdes =
+jsonFormat(UserLogEntry.apply, "message", "ALCH_TENANT_ID", "stream_str",
"time_date", "action_str")
+}
+
+case class ElasticSearchLogStoreConfig(protocol: String,
+ host: String,
+ port: Int,
+ logMessageField: String,
+ activationIdField: String,
+ requiredHeaders: String)
Review comment:
Was having trouble here when overriding pureconfig with Ansible. Seems
Ansible is not providing the same list format that pureconfig expects. As a
result, error occur when `requiredHeaders` type is `List[String`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] dubeejw commented on a change in pull request #3421: Add ElasticSearch Log Store
dubeejw commented on a change in pull request #3421: Add ElasticSearch Log Store
URL:
https://github.com/apache/incubator-openwhisk/pull/3421#discussion_r173701109
##
File path:
common/scala/src/main/scala/whisk/core/containerpool/logging/ElasticSearchLogStore.scala
##
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.logging
+
+import java.nio.file.{Path, Paths}
+import java.time.LocalDate
+import java.time.format.DateTimeFormatter
+
+import akka.actor.ActorSystem
+import akka.http.scaladsl.model.HttpRequest
+import akka.http.scaladsl.model.Uri
+import akka.stream.scaladsl.Flow
+import akka.http.scaladsl.model.HttpResponse
+import akka.http.scaladsl.model._
+
+import whisk.common.AkkaLogging
+import whisk.core.entity.{ActivationLogs, Identity, WhiskActivation}
+import whisk.core.containerpool.logging.ElasticSearchJsonProtocol._
+import whisk.core.ConfigKeys
+
+import scala.concurrent.Future
+import scala.concurrent.Promise
+import scala.util.Try
+
+import spray.json._
+
+import pureconfig._
+
+// Schema of logs in ES
+case class UserLogEntry(message: String, tenantId: String, stream: String,
time: String, action: String)
+
+object UserLogEntry extends DefaultJsonProtocol {
+ implicit val serdes =
+jsonFormat(UserLogEntry.apply, "message", "ALCH_TENANT_ID", "stream_str",
"time_date", "action_str")
+}
+
+case class ElasticSearchLogStoreConfig(protocol: String,
+ host: String,
+ port: Int,
+ logMessageField: String,
+ activationIdField: String,
+ requiredHeaders: String)
Review comment:
Was having trouble here when overriding pureconfig with Ansible. Seems
Ansible is not providing the same list format that pureconfig expects. As a
result, an error occurs when `requiredHeaders` type is `List[String`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] dubeejw commented on a change in pull request #3421: Add ElasticSearch Log Store
dubeejw commented on a change in pull request #3421: Add ElasticSearch Log Store
URL:
https://github.com/apache/incubator-openwhisk/pull/3421#discussion_r173701109
##
File path:
common/scala/src/main/scala/whisk/core/containerpool/logging/ElasticSearchLogStore.scala
##
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.logging
+
+import java.nio.file.{Path, Paths}
+import java.time.LocalDate
+import java.time.format.DateTimeFormatter
+
+import akka.actor.ActorSystem
+import akka.http.scaladsl.model.HttpRequest
+import akka.http.scaladsl.model.Uri
+import akka.stream.scaladsl.Flow
+import akka.http.scaladsl.model.HttpResponse
+import akka.http.scaladsl.model._
+
+import whisk.common.AkkaLogging
+import whisk.core.entity.{ActivationLogs, Identity, WhiskActivation}
+import whisk.core.containerpool.logging.ElasticSearchJsonProtocol._
+import whisk.core.ConfigKeys
+
+import scala.concurrent.Future
+import scala.concurrent.Promise
+import scala.util.Try
+
+import spray.json._
+
+import pureconfig._
+
+// Schema of logs in ES
+case class UserLogEntry(message: String, tenantId: String, stream: String,
time: String, action: String)
+
+object UserLogEntry extends DefaultJsonProtocol {
+ implicit val serdes =
+jsonFormat(UserLogEntry.apply, "message", "ALCH_TENANT_ID", "stream_str",
"time_date", "action_str")
+}
+
+case class ElasticSearchLogStoreConfig(protocol: String,
+ host: String,
+ port: Int,
+ logMessageField: String,
+ activationIdField: String,
+ requiredHeaders: String)
Review comment:
Was having trouble here when overriding pureconfig with Ansible. Seems
Ansible is not providing the same list format that pureconfig expects. As a
result, an error occur when `requiredHeaders` type is `List[String`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] dubeejw commented on a change in pull request #3421: Add ElasticSearch Log Store
dubeejw commented on a change in pull request #3421: Add ElasticSearch Log Store
URL:
https://github.com/apache/incubator-openwhisk/pull/3421#discussion_r173701109
##
File path:
common/scala/src/main/scala/whisk/core/containerpool/logging/ElasticSearchLogStore.scala
##
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.logging
+
+import java.nio.file.{Path, Paths}
+import java.time.LocalDate
+import java.time.format.DateTimeFormatter
+
+import akka.actor.ActorSystem
+import akka.http.scaladsl.model.HttpRequest
+import akka.http.scaladsl.model.Uri
+import akka.stream.scaladsl.Flow
+import akka.http.scaladsl.model.HttpResponse
+import akka.http.scaladsl.model._
+
+import whisk.common.AkkaLogging
+import whisk.core.entity.{ActivationLogs, Identity, WhiskActivation}
+import whisk.core.containerpool.logging.ElasticSearchJsonProtocol._
+import whisk.core.ConfigKeys
+
+import scala.concurrent.Future
+import scala.concurrent.Promise
+import scala.util.Try
+
+import spray.json._
+
+import pureconfig._
+
+// Schema of logs in ES
+case class UserLogEntry(message: String, tenantId: String, stream: String,
time: String, action: String)
+
+object UserLogEntry extends DefaultJsonProtocol {
+ implicit val serdes =
+jsonFormat(UserLogEntry.apply, "message", "ALCH_TENANT_ID", "stream_str",
"time_date", "action_str")
+}
+
+case class ElasticSearchLogStoreConfig(protocol: String,
+ host: String,
+ port: Int,
+ logMessageField: String,
+ activationIdField: String,
+ requiredHeaders: String)
Review comment:
Was having trouble here when overriding pureconfig with Ansible. Seems
Ansible is not providing the same list format that pureconfig expects. As a
result, an error occurs when `requiredHeaders` type is `List[String]`. For now
I am just splitting the string by commas.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] dubeejw commented on a change in pull request #3421: Add ElasticSearch Log Store
dubeejw commented on a change in pull request #3421: Add ElasticSearch Log Store
URL:
https://github.com/apache/incubator-openwhisk/pull/3421#discussion_r173701666
##
File path:
common/scala/src/main/scala/whisk/core/containerpool/logging/ElasticSearchLogStore.scala
##
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.logging
+
+import java.nio.file.{Path, Paths}
+import java.time.LocalDate
+import java.time.format.DateTimeFormatter
+
+import akka.actor.ActorSystem
+import akka.http.scaladsl.model.HttpRequest
+import akka.http.scaladsl.model.Uri
+import akka.stream.scaladsl.Flow
+import akka.http.scaladsl.model.HttpResponse
+import akka.http.scaladsl.model._
+
+import whisk.common.AkkaLogging
+import whisk.core.entity.{ActivationLogs, Identity, WhiskActivation}
+import whisk.core.containerpool.logging.ElasticSearchJsonProtocol._
+import whisk.core.ConfigKeys
+
+import scala.concurrent.Future
+import scala.concurrent.Promise
+import scala.util.Try
+
+import spray.json._
+
+import pureconfig._
+
+// Schema of logs in ES
+case class UserLogEntry(message: String, tenantId: String, stream: String,
time: String, action: String)
+
+object UserLogEntry extends DefaultJsonProtocol {
+ implicit val serdes =
+jsonFormat(UserLogEntry.apply, "message", "ALCH_TENANT_ID", "stream_str",
"time_date", "action_str")
+}
+
+case class ElasticSearchLogStoreConfig(protocol: String,
+ host: String,
+ port: Int,
+ logMessageField: String,
+ activationIdField: String,
+ requiredHeaders: String)
+
+/**
+ * Docker based implementation of a LogStore.
+ *
+ * Relies on docker's implementation details with regards to the JSON
log-driver. When using the JSON log-driver
+ * docker writes stdout/stderr to a JSON formatted file which is read by this
store. Logs are written in the
+ * activation record itself.
+ *
+ * Additionally writes logs to a separate file which can be processed by any
backend service asynchronously.
+ */
+class ElasticSearchLogStore(
+ system: ActorSystem,
+ httpFlow: Option[Flow[(HttpRequest, Promise[HttpResponse]),
(Try[HttpResponse], Promise[HttpResponse]), Any]] = None,
+ destinationDirectory: Path = Paths.get("logs"),
+ elasticSearchConfig: ElasticSearchLogStoreConfig =
+loadConfigOrThrow[ElasticSearchLogStoreConfig](ConfigKeys.elasticSearch))
+extends DockerToActivationFileLogStore(system, destinationDirectory) {
+
+ implicit val actorSystem = system
+ implicit val logging = new AkkaLogging(system.log)
+
+ val protocol = elasticSearchConfig.protocol
+ val host = elasticSearchConfig.host
+ val port = elasticSearchConfig.port
+ val logMessageField = elasticSearchConfig.logMessageField
+ val activationIdField = elasticSearchConfig.activationIdField
+ val requiredHeaders = elasticSearchConfig.requiredHeaders.split(",")
+ val esClient = new ElasticSearchRestClient(protocol, host, port, httpFlow)
+
+ private def transcribeLogs(queryResult: EsSearchResult): ActivationLogs = {
+val logs = queryResult.hits.hits.map(hit => {
+ val userLogEntry = hit.source.convertTo[UserLogEntry]
+ s"${userLogEntry.time} ${userLogEntry.stream}:
${userLogEntry.message.stripLineEnd}"
+})
+
+ActivationLogs(logs)
+ }
+
+ private def extractRequiredHeaders(request: HttpRequest) =
+request.headers.filter {
+ case header: HttpHeader if
requiredHeaders.contains(header.lowercaseName) => true
+ case _
=> false
+}.toList
+
+ override def fetchLogs(user: Identity, activation: WhiskActivation, request:
HttpRequest): Future[ActivationLogs] = {
+val headers = extractRequiredHeaders(request)
+
+if (headers.length == requiredHeaders.length) {
+ val date =
LocalDate.now.format(DateTimeFormatter.ofPattern(".MM.dd"))
+ val path =
Uri(s"/elasticsearch/logstash-${user.uuid.asString}-$date/_search")
+ val logQuery = s"_type: $logMessageField AND $activationIdField:
${ac
[GitHub] dubeejw commented on a change in pull request #3421: Add ElasticSearch Log Store
dubeejw commented on a change in pull request #3421: Add ElasticSearch Log Store
URL:
https://github.com/apache/incubator-openwhisk/pull/3421#discussion_r173703650
##
File path:
common/scala/src/main/scala/whisk/core/containerpool/logging/ElasticSearchLogStore.scala
##
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.containerpool.logging
+
+import java.nio.file.{Path, Paths}
+import java.time.LocalDate
+import java.time.format.DateTimeFormatter
+
+import akka.actor.ActorSystem
+import akka.http.scaladsl.model.HttpRequest
+import akka.http.scaladsl.model.Uri
+import akka.stream.scaladsl.Flow
+import akka.http.scaladsl.model.HttpResponse
+import akka.http.scaladsl.model._
+
+import whisk.common.AkkaLogging
+import whisk.core.entity.{ActivationLogs, Identity, WhiskActivation}
+import whisk.core.containerpool.logging.ElasticSearchJsonProtocol._
+import whisk.core.ConfigKeys
+
+import scala.concurrent.Future
+import scala.concurrent.Promise
+import scala.util.Try
+
+import spray.json._
+
+import pureconfig._
+
+// Schema of logs in ES
+case class UserLogEntry(message: String, tenantId: String, stream: String,
time: String, action: String)
+
+object UserLogEntry extends DefaultJsonProtocol {
+ implicit val serdes =
+jsonFormat(UserLogEntry.apply, "message", "ALCH_TENANT_ID", "stream_str",
"time_date", "action_str")
+}
+
+case class ElasticSearchLogStoreConfig(protocol: String,
+ host: String,
+ port: Int,
+ logMessageField: String,
+ activationIdField: String,
+ requiredHeaders: String)
+
+/**
+ * Docker based implementation of a LogStore.
+ *
+ * Relies on docker's implementation details with regards to the JSON
log-driver. When using the JSON log-driver
+ * docker writes stdout/stderr to a JSON formatted file which is read by this
store. Logs are written in the
+ * activation record itself.
+ *
+ * Additionally writes logs to a separate file which can be processed by any
backend service asynchronously.
+ */
+class ElasticSearchLogStore(
+ system: ActorSystem,
+ httpFlow: Option[Flow[(HttpRequest, Promise[HttpResponse]),
(Try[HttpResponse], Promise[HttpResponse]), Any]] = None,
+ destinationDirectory: Path = Paths.get("logs"),
+ elasticSearchConfig: ElasticSearchLogStoreConfig =
+loadConfigOrThrow[ElasticSearchLogStoreConfig](ConfigKeys.elasticSearch))
+extends DockerToActivationFileLogStore(system, destinationDirectory) {
+
+ implicit val actorSystem = system
+ implicit val logging = new AkkaLogging(system.log)
+
+ private val protocol = elasticSearchConfig.protocol
+ private val host = elasticSearchConfig.host
+ private val port = elasticSearchConfig.port
+ private val logMessageField = elasticSearchConfig.logMessageField
+ private val activationIdField = elasticSearchConfig.activationIdField
+ private val requiredHeaders = elasticSearchConfig.requiredHeaders.split(",")
+ private val esClient = new ElasticSearchRestClient(protocol, host, port,
httpFlow)
+
+ private def transcribeLogs(queryResult: EsSearchResult): ActivationLogs = {
+val logs = queryResult.hits.hits.map(hit => {
+ val userLogEntry = hit.source.convertTo[UserLogEntry]
+ s"${userLogEntry.time} ${userLogEntry.stream}:
${userLogEntry.message.stripLineEnd}"
+})
+
+ActivationLogs(logs)
+ }
+
+ private def extractRequiredHeaders(request: HttpRequest) =
+request.headers.filter {
+ case header: HttpHeader if
requiredHeaders.contains(header.lowercaseName) => true
+ case _
=> false
+}.toList
+
+ private def generatePayload(activation: WhiskActivation) = {
+val logQuery = s"_type: $logMessageField AND $activationIdField:
${activation.activationId}"
+val queryString = EsQueryString(EsQueryValue(logQuery))
+val sort = Some(Array(EsQueryTimestamp(EsQueryOrder("asc")).toJson))
Review comment:
Would be nice to not have to call `toJson` for the arguments here and below.
