A B C D E F G H I L M N O P R S T U V W X Z

A

add(Entry) - Method in class org.zephyr.data.Record

Add a non-null entry to the record

add(Entry...) - Method in class org.zephyr.data.Record

Add 1..N entries to the record, in order

AlwaysValidValidator - Class in org.zephyr.schema.validator

AlwaysValidValidator() - Constructor for class org.zephyr.schema.validator.AlwaysValidValidator

B

BasicOutputFormatter - Class in org.zephyr.output.formatter

A raw(ish) look at a record, primarily for debugging purposes.

BasicOutputFormatter() - Constructor for class org.zephyr.output.formatter.BasicOutputFormatter

C

CatalogService - Interface in org.zephyr.service

checkOutputSpecs(FileSystem, JobConf) - Method in class org.zephyr.mapreduce.ZephyrOutputFormat

configure(JobConf) - Method in class org.zephyr.mapreduce.mapper.ZephyrMapper

CsvParser - Class in org.zephyr.parser

CsvParser(List<String>, InputStream, char, char, char, boolean, boolean) - Constructor for class org.zephyr.parser.CsvParser

CsvParserFactory - Class in org.zephyr.parser

CsvParserFactory(List<String>) - Constructor for class org.zephyr.parser.CsvParserFactory

D

DateFormatNormalizer - Class in org.zephyr.schema.normalizer

DateFormatNormalizer(String, String) - Constructor for class org.zephyr.schema.normalizer.DateFormatNormalizer

If no timezone information is in incomingFormat than the systems default TimeZone will be used.

DecimalDegreesLatitudeValidator - Class in org.zephyr.schema.validator

DecimalDegreesLatitudeValidator() - Constructor for class org.zephyr.schema.validator.DecimalDegreesLatitudeValidator

DecimalDegreesLongitudeValidator - Class in org.zephyr.schema.validator

DecimalDegreesLongitudeValidator() - Constructor for class org.zephyr.schema.validator.DecimalDegreesLongitudeValidator

DEFAULT_CACHE_FOLDER_NAME - Static variable in class org.zephyr.mapreduce.ZephyrDriver

DEFAULT_ESCAPE_CHAR - Static variable in class org.zephyr.parser.CsvParserFactory

DEFAULT_QUOTE_CHAR - Static variable in class org.zephyr.parser.CsvParserFactory

DEFAULT_SEPARATOR - Static variable in class org.zephyr.parser.CsvParserFactory

DEPTH_FIRST - Static variable in class org.zephyr.parser.XMLParserFactory

DepthFirstXMLParser - Class in org.zephyr.parser

This DepthFirstXMLParser uses a STAX streaming implementation to parse the provided InputStream.

DepthFirstXMLParser(String, InputStream) - Constructor for class org.zephyr.parser.DepthFirstXMLParser

DoubleValidator - Class in org.zephyr.schema.validator

DoubleValidator() - Constructor for class org.zephyr.schema.validator.DoubleValidator

E

enrich(Record) - Method in interface org.zephyr.enricher.Enricher

The enrich method MUST either: - Leave the original record as it was in case of un-enrichable data or - return the modified record on success or - throw a RuntimeException in case of exceptional circumstance (exception is logged and processing continues)

Enricher - Interface in org.zephyr.enricher

Enricher contract for Zephyr

Entry - Class in org.zephyr.data

An Entry is a value with a label, a value (usually normalized and validated), a list of types, a visibility field for analytics or databases that might require it (depending on which analytic or database it is, it may require some form of conversion), and a metadata field.

Entry(String, String, List<String>, String, String) - Constructor for class org.zephyr.data.Entry

Fully qualified constructor for an Entry

F

formatRecord(Record) - Method in class org.zephyr.output.formatter.BasicOutputFormatter

formatRecord(Record) - Method in class org.zephyr.output.formatter.HiveOutputFormatter

Builds a HashMap lookup for all of the entries in a Record, then iterates through the given orderedHeaders as a list of key lookups.

formatRecord(Record) - Method in interface org.zephyr.output.formatter.OutputFormatter

G

generateActualKey(Text, Text) - Method in class org.zephyr.mapreduce.ZephyrOutputFormat

generateFileNameForKeyValue(Text, Text, String) - Method in class org.zephyr.mapreduce.ZephyrOutputFormat

generateUUID() - Static method in class org.zephyr.util.UUIDHelper

Generates a time based UUID using the mac address of the machine, if available

get(int) - Method in class org.zephyr.data.Record

getConfigMap() - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

getEnrichers() - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

getEntry(String, String, List<String>, String, String) - Method in interface org.zephyr.service.CatalogService

getError() - Method in class org.zephyr.data.ProcessingResult

getFeedName() - Method in class org.zephyr.data.Record

getFeedName() - Method in class org.zephyr.schema.Schema

getIncomingFormat() - Method in class org.zephyr.schema.normalizer.DateFormatNormalizer

getIncomingLocale() - Method in class org.zephyr.schema.normalizer.DateFormatNormalizer

getInputFileBasedOutputFileName(JobConf, String) - Method in class org.zephyr.mapreduce.ZephyrOutputFormat

getInputFormat() - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

getInputPath() - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

getInputStreamFromKeyValue(NullWritable, BytesWritable) - Method in class org.zephyr.mapreduce.mapper.ZephyrBytesWritableMapper

getInputStreamFromKeyValue(K, V) - Method in class org.zephyr.mapreduce.mapper.ZephyrMapper

getInputStreamFromKeyValue(LongWritable, Text) - Method in class org.zephyr.mapreduce.mapper.ZephyrTextMapper

getJobName() - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

getKey() - Method in class org.zephyr.data.Pair

getLabel() - Method in class org.zephyr.data.Entry

getMapper() - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

getMetadata() - Method in class org.zephyr.data.Entry

getOutgoingFormat() - Method in class org.zephyr.schema.normalizer.DateFormatNormalizer

getOutgoingLocale() - Method in class org.zephyr.schema.normalizer.DateFormatNormalizer

getOutputFormatter() - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

getOutputPath() - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

getParserFactory() - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

getPreprocessor() - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

getProcessedData() - Method in class org.zephyr.data.ProcessingResult

getRawData() - Method in class org.zephyr.data.ProcessingResult

getRawFieldName() - Method in class org.zephyr.schema.scheme.Scheme

getRecordReader(InputSplit, JobConf, Reporter) - Method in class org.zephyr.mapreduce.WholeFileInputFormat

getSchema() - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

getSourceElementIdentifier() - Method in class org.zephyr.parser.Parser

getSourceElementPair() - Method in class org.zephyr.parser.Parser

getTimezone() - Method in class org.zephyr.schema.normalizer.DateFormatNormalizer

getTypes() - Method in class org.zephyr.data.Entry

getUuid() - Method in class org.zephyr.data.Record

getValue() - Method in class org.zephyr.data.Entry

getValue() - Method in class org.zephyr.data.Pair

getVisibility() - Method in class org.zephyr.data.Entry

H

HiveOutputFormatter - Class in org.zephyr.output.formatter

The HiveOutputFormatter is a class that maps a given record to the column order we have been given.

HiveOutputFormatter(List<String>) - Constructor for class org.zephyr.output.formatter.HiveOutputFormatter

I

initialize() - Method in class org.zephyr.output.Outputter

IntegerValidator - Class in org.zephyr.schema.validator

IntegerValidator() - Constructor for class org.zephyr.schema.validator.IntegerValidator

ISO_8601_FORMAT - Static variable in class org.zephyr.schema.normalizer.DateFormatNormalizer

ISO_8601_FORMAT - Static variable in class org.zephyr.schema.normalizer.UnixDateFormatNormalizer

IsoDateFormatValidator - Class in org.zephyr.schema.validator

IsoDateFormatValidator() - Constructor for class org.zephyr.schema.validator.IsoDateFormatValidator

isSplitable(FileSystem, Path) - Method in class org.zephyr.mapreduce.WholeFileInputFormat

isValid(String) - Method in class org.zephyr.schema.validator.AlwaysValidValidator

isValid(String) - Method in class org.zephyr.schema.validator.DecimalDegreesLatitudeValidator

isValid(String) - Method in class org.zephyr.schema.validator.DecimalDegreesLongitudeValidator

isValid(String) - Method in class org.zephyr.schema.validator.DoubleValidator

isValid(String) - Method in class org.zephyr.schema.validator.IntegerValidator

isValid(String) - Method in class org.zephyr.schema.validator.IsoDateFormatValidator

isValid(String) - Method in class org.zephyr.schema.validator.LongValidator

isValid(String) - Method in class org.zephyr.schema.validator.NonEmptyValidator

isValid(String) - Method in interface org.zephyr.schema.validator.Validator

iterator() - Method in class org.zephyr.data.Record

Allows us to iterate over the array list of entries this record is composed of

L

LongValidator - Class in org.zephyr.schema.validator

LongValidator() - Constructor for class org.zephyr.schema.validator.LongValidator

M

main(String[]) - Static method in class org.zephyr.mapreduce.ZephyrDriver

map(K, V, OutputCollector<Text, Text>, Reporter) - Method in class org.zephyr.mapreduce.mapper.ZephyrMapper

map(List<Pair>) - Method in class org.zephyr.schema.Schema

This method takes in a List of orderedPairings (key:value pairs).

map(Pair, CatalogService) - Method in class org.zephyr.schema.scheme.Scheme

The map method is called by the Schema's normalizeAndValidate() method - it takes in a Pair object, validates it against the preNormalizationValidator, executes all normalizers on the value (in order, as applicable), and then validates it against the postNormalizationValidator.

N

newParser(InputStream) - Method in class org.zephyr.parser.CsvParserFactory

newParser(InputStream) - Method in interface org.zephyr.parser.ParserFactory

newParser(InputStream) - Method in class org.zephyr.parser.XMLParserFactory

NonEmptyValidator - Class in org.zephyr.schema.validator

NonEmptyValidator() - Constructor for class org.zephyr.schema.validator.NonEmptyValidator

NormalizationException - Exception in org.zephyr.schema.normalizer

NormalizationException() - Constructor for exception org.zephyr.schema.normalizer.NormalizationException

NormalizationException(String) - Constructor for exception org.zephyr.schema.normalizer.NormalizationException

NormalizationException(Throwable) - Constructor for exception org.zephyr.schema.normalizer.NormalizationException

NormalizationException(String, Throwable) - Constructor for exception org.zephyr.schema.normalizer.NormalizationException

normalize(String) - Method in class org.zephyr.schema.normalizer.DateFormatNormalizer

normalize(String) - Method in interface org.zephyr.schema.normalizer.Normalizer

normalize(String) - Method in class org.zephyr.schema.normalizer.ReplaceAllNormalizer

normalize(String) - Method in class org.zephyr.schema.normalizer.SplitNormalizer

normalize(String) - Method in class org.zephyr.schema.normalizer.UnixDateFormatNormalizer

normalizeAndValidate(Pair) - Method in class org.zephyr.schema.Schema

Attempts to find a Schema for the provided Pair (by using the Pair.getKey() as the key for the member variable "mappings").

Normalizer - Interface in org.zephyr.schema.normalizer

O

org.zephyr.data - package org.zephyr.data

org.zephyr.enricher - package org.zephyr.enricher

org.zephyr.mapreduce - package org.zephyr.mapreduce

org.zephyr.mapreduce.mapper - package org.zephyr.mapreduce.mapper

org.zephyr.output - package org.zephyr.output

org.zephyr.output.formatter - package org.zephyr.output.formatter

org.zephyr.parser - package org.zephyr.parser

org.zephyr.preprocessor - package org.zephyr.preprocessor

org.zephyr.schema - package org.zephyr.schema

org.zephyr.schema.normalizer - package org.zephyr.schema.normalizer

org.zephyr.schema.scheme - package org.zephyr.schema.scheme

org.zephyr.schema.validator - package org.zephyr.schema.validator

org.zephyr.service - package org.zephyr.service

org.zephyr.util - package org.zephyr.util

output(Record) - Method in class org.zephyr.output.Outputter

OutputFormatter - Interface in org.zephyr.output.formatter

An OutputFormatter class converts a Record into the format we want to write to disk in the case of any sort of output operation wherein we write to disk

It can either be used by a Writer class or directly.

Outputter - Class in org.zephyr.output

Outputter() - Constructor for class org.zephyr.output.Outputter

P

Pair - Class in org.zephyr.data

A key-value pairing of data

Pair(String, String) - Constructor for class org.zephyr.data.Pair

Constructor that takes in the key and value

parse() - Method in class org.zephyr.parser.CsvParser

parse() - Method in class org.zephyr.parser.DepthFirstXMLParser

The parse method will parse the XML document, flattening it, and turning the path down to the QName

parse() - Method in class org.zephyr.parser.Parser

The parse method will parse the next list of key:value pairs out of the

ParseException - Exception in org.zephyr.parser

ParseException() - Constructor for exception org.zephyr.parser.ParseException

ParseException(String) - Constructor for exception org.zephyr.parser.ParseException

ParseException(Throwable) - Constructor for exception org.zephyr.parser.ParseException

ParseException(String, Throwable) - Constructor for exception org.zephyr.parser.ParseException

Parser - Class in org.zephyr.parser

Parser contract for Zephyr

Parser() - Constructor for class org.zephyr.parser.Parser

ParserFactory - Interface in org.zephyr.parser

ParserFactory contract for Zephyr; an implementation of ParserFactory will likely have Parser configuration specific parameters specified on them.

Preprocessor - Interface in org.zephyr.preprocessor

process(InputStream) - Method in interface org.zephyr.preprocessor.Preprocessor

ProcessingResult<T,R> - Class in org.zephyr.data

We have two primary steps in Zephyr that could result in either a successful operation or exceptional behavior.

ProcessingResult(T) - Constructor for class org.zephyr.data.ProcessingResult

Constructor for properly processed data

ProcessingResult(R, Throwable) - Constructor for class org.zephyr.data.ProcessingResult

Constructor for a failed processing step

R

Record - Class in org.zephyr.data

An record of data that contains 0..N Entries, in order, and provides accessor methods to them.

Record() - Constructor for class org.zephyr.data.Record

No argument constructor that specifies a newly generated UUID and an empty feed name for this record

Record(String, String) - Constructor for class org.zephyr.data.Record

Creates a new Record object with the provided UUID and feedName (and sets the list of entries to be a new ArrayList)

ReplaceAllNormalizer - Class in org.zephyr.schema.normalizer

ReplaceAllNormalizer(String, String) - Constructor for class org.zephyr.schema.normalizer.ReplaceAllNormalizer

run(String[]) - Method in class org.zephyr.mapreduce.ZephyrDriver

S

Schema - Class in org.zephyr.schema

The Schema abstract class provides implementing classes with a contract to follow and some standard methods defined for the Scheme, Normalization, and Validation of a key:value pair for each implementing class (and subsequent feed).

Schema() - Constructor for class org.zephyr.schema.Schema

Scheme - Class in org.zephyr.schema.scheme

A scheme defines a mapping between the raw field name to the outgoing entry label, the visibility string to utilize, metadata, and what type(s) that should be applied to the scheme for typing in later enrichers or outputters

Scheme(String, String) - Constructor for class org.zephyr.schema.scheme.Scheme

Constructor that requires the raw field name and the label, sets both validators as NON_EMPTY_VALIDATORs, and sets the Normalizer and Types lists to empty (and immutable) lists.

SchemeException - Exception in org.zephyr.schema.scheme

SchemeException() - Constructor for exception org.zephyr.schema.scheme.SchemeException

SchemeException(String) - Constructor for exception org.zephyr.schema.scheme.SchemeException

SchemeException(Throwable) - Constructor for exception org.zephyr.schema.scheme.SchemeException

SchemeException(String, Throwable) - Constructor for exception org.zephyr.schema.scheme.SchemeException

setCatalogService(CatalogService) - Method in class org.zephyr.schema.Schema

setConfigMap(Map<String, String>) - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

setEnrichers(List<Enricher>) - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

setEscapeChar(char) - Method in class org.zephyr.parser.CsvParserFactory

setFeedName(String) - Method in class org.zephyr.data.Record

setFeedName(String) - Method in class org.zephyr.schema.Schema

setFieldSeparator(char) - Method in class org.zephyr.output.formatter.HiveOutputFormatter

setIncomingLocale(Locale) - Method in class org.zephyr.schema.normalizer.DateFormatNormalizer

setInputFormat(InputFormat<? extends Writable, ? extends Writable>) - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

setInputPath(String) - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

setJobName(String) - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

setMapper(Mapper<? extends Writable, ? extends Writable, ? extends Writable, ? extends Writable>) - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

setMetadata(String) - Method in class org.zephyr.data.Entry

setMetadata(String) - Method in class org.zephyr.schema.scheme.Scheme

setNormalizers(List<Normalizer>) - Method in class org.zephyr.schema.scheme.Scheme

setOutgoingLocale(Locale) - Method in class org.zephyr.schema.normalizer.DateFormatNormalizer

setOutputFormatter(OutputFormatter) - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

setOutputPath(String) - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

setParserFactory(ParserFactory) - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

setParserType(String) - Method in class org.zephyr.parser.XMLParserFactory

setPostNormalizationValidator(Validator) - Method in class org.zephyr.schema.scheme.Scheme

setPreNormalizationValidator(Validator) - Method in class org.zephyr.schema.scheme.Scheme

setPreprocessor(Preprocessor) - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

setQuoteChar(char) - Method in class org.zephyr.parser.CsvParserFactory

setRequiredFields(Set<String>) - Method in class org.zephyr.schema.Schema

setSchema(Schema) - Method in class org.zephyr.mapreduce.ZephyrMRv1Configuration

setSchemata(List<Scheme>) - Method in class org.zephyr.schema.Schema

setSeparator(char) - Method in class org.zephyr.parser.CsvParserFactory

setSourceElementIdentifier(String) - Method in class org.zephyr.parser.Parser

setStrictQuotes(boolean) - Method in class org.zephyr.parser.CsvParserFactory

setTimeZone(String) - Method in class org.zephyr.schema.normalizer.DateFormatNormalizer

setTrimWhitespace(boolean) - Method in class org.zephyr.parser.CsvParserFactory

setTypes(List<String>) - Method in class org.zephyr.schema.scheme.Scheme

setVisibility(String) - Method in class org.zephyr.schema.scheme.Scheme

size() - Method in class org.zephyr.data.Record

SOURCE_ELEMENT_IDENTIFIER_HEADER - Static variable in class org.zephyr.parser.Parser

SplitNormalizer - Class in org.zephyr.schema.normalizer

SplitNormalizer(String, int) - Constructor for class org.zephyr.schema.normalizer.SplitNormalizer

T

toString() - Method in class org.zephyr.data.Pair

Human readable toString override.

U

UnixDateFormatNormalizer - Class in org.zephyr.schema.normalizer

UnixDateFormatNormalizer(String) - Constructor for class org.zephyr.schema.normalizer.UnixDateFormatNormalizer

If no timezone information is in incomingFormat than the systems default TimeZone will be used.

UnixDateFormatNormalizer(TimeZone, String) - Constructor for class org.zephyr.schema.normalizer.UnixDateFormatNormalizer

UUIDHelper - Class in org.zephyr.util

Purpose: Utilities to facilitate UUID generation

UUIDHelper() - Constructor for class org.zephyr.util.UUIDHelper

V

Validator - Interface in org.zephyr.schema.validator

W

wasProcessedSuccessfully() - Method in class org.zephyr.data.ProcessingResult

WholeFileInputFormat - Class in org.zephyr.mapreduce

WholeFileInputFormat() - Constructor for class org.zephyr.mapreduce.WholeFileInputFormat

X

XMLParserFactory - Class in org.zephyr.parser

XMLParserFactory can provide either a depth first, generalized DepthFirstXMLParser for streaming purposes, or (eventually) a breadth first, DOM based parser (for instances where "repeating" values appear "before" the global values and a less efficient, less big-data-ish approach must be used to parse the data out).

XMLParserFactory(String) - Constructor for class org.zephyr.parser.XMLParserFactory

Z

ZephyrBytesWritableMapper - Class in org.zephyr.mapreduce.mapper

ZephyrBytesWritableMapper() - Constructor for class org.zephyr.mapreduce.mapper.ZephyrBytesWritableMapper

ZephyrDriver - Class in org.zephyr.mapreduce

ZephyrDriver(ZephyrMRv1Configuration, String) - Constructor for class org.zephyr.mapreduce.ZephyrDriver

ZephyrMapper<K,V> - Class in org.zephyr.mapreduce.mapper

ZephyrMapper() - Constructor for class org.zephyr.mapreduce.mapper.ZephyrMapper

ZephyrMRv1Configuration - Class in org.zephyr.mapreduce

ZephyrMRv1Configuration() - Constructor for class org.zephyr.mapreduce.ZephyrMRv1Configuration

ZephyrOutputFormat - Class in org.zephyr.mapreduce

ZephyrOutputFormat() - Constructor for class org.zephyr.mapreduce.ZephyrOutputFormat

ZephyrTextMapper - Class in org.zephyr.mapreduce.mapper

ZephyrTextMapper() - Constructor for class org.zephyr.mapreduce.mapper.ZephyrTextMapper

A B C D E F G H I L M N O P R S T U V W X Z