test-hdfs

command
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 10, 2020 License: Apache-2.0 Imports: 9 Imported by: 0

README

Integration Test

This directory contains tests to be executed against a running Hadoop File System.

Instructions

The following assumes your GOPATH is setup properly.

  • Start your local Hadoop File System or make sure to have access to a remote running system.
  • Ensure your HDFS system has dfs.webhdfs.enabled property enabled in hsdfs-site.xml
  • Import gowfs with: go get github.com/vladimirvivien/gowfs
  • Don't forget to pull down the actual gowfs project.
  • Change your working directory to gowfs/test-hdfs
  • curl/download file http://www.gutenberg.org/cache/epub/2600/pg2600.txt
  • Save the downloaded file as war-and-peace.txt in the working directory.
  • Build using go build. This will create test-hdfs binary.

HDFS Setup

  • Enable dfs.webhdfs.enabled property in your hsdfs-site.xml
  • Ensure hadoop.http.staticuser.user property is set in your core-site.xml.

Run Test

./test-hdfs -namenode=<your namenode addr> -path=<hdfs path> -user=<your hdfs user>

What Test Does

The main.go code will connect to the specified namenode and run several steps including

  • Create a test directory in -path
  • Upload file from above
  • Rename the file
  • Append to the file
  • Change owner, group, and mode.
  • Download the file and remove it from HDFS.

Documentation

The Go Gopher

There is no documentation for this package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL