[
https://issues.apache.org/jira/browse/BIGTOP-952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13916754#comment-13916754
]
jay vyas commented on BIGTOP-952:
---------------------------------
Okay, made some progress on this. Heres a snippet of the WIP code for
provisioning from JSON (FYI I have to switch the { to ['s in the BIGTOP-1200,
{noformat}
import groovy.json.JsonSlurper;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.util.List;
import java.util.Map;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.permission.FsPermission;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.Writer;
def uri = "hdfs://"
def v = new JsonSlurper();
def jsonParser = new JsonSlurper();
def json = "init-hcfs.json";
def parsedData = jsonParser.parse( new FileReader(json));
def dirs = (List) parsedData.get("dir");
Configuration conf = new Configuration();
//Configuration.dumpConfiguration(conf, new java.io.PrintWriter(System.out));
FileSystem fs = FileSystem.get(conf);
System.out.println(fs.getClass());
System.out.println("DIRS : " + dirs + " DIRSSSS");
dirs.each() {
println(it);
name=it[0];
mode=it[1];
user=it[2];
group=it[3];
Path file = new Path(uri+name);
System.out.println("mkdirs " + name + " " + mode);
System.out.println("Ownder " + name + " " + user + " " + group);
//fs.mkdirs(name, new FsPermission(Short.parseShort(mode));
//fs.setOwner(name, user, group);
}
{noformat}
> init-hdfs.sh is dog slow. Let's replace it with a single VM call and better
> layout management
> ---------------------------------------------------------------------------------------------
>
> Key: BIGTOP-952
> URL: https://issues.apache.org/jira/browse/BIGTOP-952
> Project: Bigtop
> Issue Type: Improvement
> Components: Deployment
> Affects Versions: 0.5.0
> Reporter: Konstantin Boudnik
> Assignee: Roman Shaposhnik
> Priority: Blocker
> Fix For: 0.8.0
>
> Attachments: untar.groovy
>
>
> As has been proposed in [this
> patch|https://issues.apache.org/jira/secure/attachment/12575644/untarHdfs.groovy]
> by [~rvs] there's a very efficient way of creating layout in HDFS using a
> tarfile and Groovy script with direct call into DFS APIs.
> Let's making it happen.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
