Hey Paul, It turns out that I got it to work by added a test dependency in the plugin's pom.xml file. I ended up keeping the unit test because in this instance, one of the fields returned is a VARBINARY field, specifically a shape, which is not really possible to interpret without using one of the decoder functions. Anyway, it's done and submitted now, although the travis-ci builds aren't working for some reason. (grr)
This is the PR in question: https://github.com/apache/drill/pull/1858. I had been watching this for some time now and was hoping someone would complete it. Once you finished the EVF, this looked really easy to complete. It wasn't really easy, but it was a good learning experience. -- C > On Sep 20, 2019, at 1:52 PM, Paul Rogers <par0...@yahoo.com.INVALID> wrote: > > Hi Charles, > > Your workaround sounds right. In general, unit tests should make as few > assumptions as possible about the system under test; they should focus on the > one part of the system they want to exercise. (There is a large body of > testing theory behind this statement...) > > In your case, it may be that the contrib project that contains your UDFs is > not a dependency of the contrib project with your format plugin. Nor should > it be. > > We have a separate set of end-to-end tests which would be a great place to, > say, make sure that all your UDFs work with all your format plugins. > Unfortunately, these can't be run by developers; you'd need help from whoever > currently owns the test framework. > > Thanks, > - Paul > > > > On Friday, September 20, 2019, 09:50:54 AM PDT, Charles Givre > <cgi...@gmail.com> wrote: > > Hey Paul, > Thanks for your help. I should have clarified that the work I'm doing is for > an ESRI Shape File plugin which is in contrib. The unit test in question > calls a function which is in the contrib/udfs and I think you've pinpointed > the issue. Unfortunately, running the test with maven produced the same > error. I may just remove the unit test in question since the others pass, > and this really doesn't have anything to do with the functionality of the > code. > -- C > >> On Sep 20, 2019, at 12:40 PM, Paul Rogers <par0...@yahoo.com.INVALID> wrote: >> >> Hi Charles, >> >> I seem to recall fighting with something similar in the past. The problem is >> not with your setup; it is with how Drill finds your (custom?) UDF on the >> classpath. >> >> My memory is hazy; but I think it had to do with the way that Drill uses the >> drill-override.conf file to extend class path scanning, and how it finds the >> UDF code. I think I banged on it until it worked, but I don't recall what I >> did. >> >> Maybe I do remember. I think that, to run your code in the IDE, you need to >> add the source code directory to your class path. (Recall that Drill needs >> access to both function source and compiled code.) I think I modified my IDE >> debug launch command to always include the proper sources. I don't have that >> config in front of me; I'll check it this weekend to see if I can find the >> exact item you must add. >> >> A workaround may be to run the test using Maven [1], since the Maven configs >> will do the needed magic: >> >> cd exec/java-exec >> mvn surefire:test -Dtest=YourTest >> >> >> The other possibility is that you have your UDF in the "contrib" project, >> while you are running unit tests in the "exec" project. Exec does not depend >> on contrib, so contrib code is not visible to unit tests in Exec. The same >> is true, by the way, for the us of the JDBC driver, since that is in the >> Maven project after exec. >> >> Thanks, >> - Paul >> >> [1] >> https://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html >> >> >> >> >> On Friday, September 20, 2019, 07:44:55 AM PDT, Charles Givre >> <cgi...@gmail.com> wrote: >> >> Hello Drillers, >> I'm encountering a strange error in a unit test. The code is included >> below, and it fails because when Drill attempts to execute the test, it >> cannot find the function st_astext(). If I build Drill and execute the >> query in the CLI it works, so I suspect there is some environment issue >> rather than a Drill issue. Does anyone have any suggestions? >> Thanks! >> >> >> @BeforeClass >> public static void setup() throws Exception { >> startCluster(ClusterFixture.builder(dirTestWatcher)); >> >> DrillbitContext context = cluster.drillbit().getContext(); >> FileSystemConfig original = (FileSystemConfig) >> context.getStorage().getPlugin("cp").getConfig(); >> Map<String, FormatPluginConfig> newFormats = new >> HashMap<>(original.getFormats()); >> newFormats.put("shp", new ShpFormatConfig()); >> FileSystemConfig pluginConfig = new >> FileSystemConfig(original.getConnection(), original.getConfig(), >> original.getWorkspaces(), newFormats); >> pluginConfig.setEnabled(true); >> context.getStorage().createOrUpdate("cp", pluginConfig, true); >> } >> ... >> >> @Test >> public void testShpQuery() throws Exception { >> >> testBuilder() >> .sqlQuery("select gid, srid, shapeType, name, st_astext(geom) as wkt " >> + "from cp.`CA-cities.shp` where gid = 100") >> .ordered() >> .baselineColumns("gid", "srid", "shapeType", "name", "wkt") >> .baselineValues(100, 4326, "Point", "Jenny Lind", "POINT (-120.8699371 >> 38.0949216)") >> .build() >> .run(); >> }
