Concatenator is a process that concatenates the content of multiple files received in the in-port In, into one file returned on its out-port, Out
NewConcatenator returns a new, initialized Concatenator process
In returns the (only) in-port for this process
Out returns the (only) out-port for this process
Run runs the Concatenator process
FileReader takes a file path on its FilePath in-port, and returns the file content as byte on its out-port Out
NewFileReader returns an initialized new FileReader
InFilePath returns the parameter in-port on which a file name is read
OutLine returns an parameter out-port with lines of the files being read
Run the FileReader
FileSource is initiated with a set of file paths, which it will send as a stream of File IPs on its outport Out()
NewFileSource returns a new initialized FileSource process
Out returns the out-port, on which file IPs based on the file paths the process was initialized with, will be retrieved.
Run runs the FileSource process
FileSplitter is a process that will split a file into multiple files, each with LinesPerSplit number of lines per file
NewFileSplitter returns an initialized FileSplitter process that will split a file into multiple files, each with linesPerSplit number of lines per file
InFile returns the port for the input file
OutSplitFile returns the resulting split (part) files generated0
Run runs the FileSplitter process
MapToKeys is a process that runs a function provided by the user, upon initialization, that will provide a map of key:value pairs, based in IPs read on the In-port. The key:value pairs (maps) are added to the IPs on the out-port, which are identical to the incoming IPs, except for the new key:value map
NewMapToKeys returns an initialized MapToKeys process
In takes input files the content of which the map function will be run, to generate keys
Out outputs files that are supplemented with keys by the map function.
Run runs the MapToKeys process
ParamSource will feed parameters on an out-port
NewParamSource returns a new ParamSource
Out returns the out-port, on which parameters the process was initialized with, will be retrieved.
Run runs the process
StreamToSubStream takes a normal stream of IP's representing individual files, and returns one IP where the incoming IPs are sent on its substream.
NewStreamToSubStream instantiates a new StreamToSubStream process
In returns the in-port
OutSubStream returns the out-port
Run runs the StreamToSubStream