ShuffledSequenceFileInputFormat (Cloud Bigtable HBase Client for Java 1.12.0 API)

com.google.cloud.bigtable.mapreduce

Class ShuffledSequenceFileInputFormat<K,V>

  • java.lang.Object
    • org.apache.hadoop.mapreduce.InputFormat<K,V>
      • org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>
        • org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat<K,V>
          • com.google.cloud.bigtable.mapreduce.ShuffledSequenceFileInputFormat<K,V>

  • public class ShuffledSequenceFileInputFormat<K,V>
extends org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat<K,V>
    An InputFormat randomizes the order of SequenceFile splits. This is needed for the Bigtable import job to spread the load across all of the Bigtable tablet servers.
    Version:
    $Id: $Id
    Author:
    igorbernstein2

    • Nested Class Summary

      • Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

        org.apache.hadoop.mapreduce.lib.input.FileInputFormat.Counter

    • Field Summary

      • Fields inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

        DEFAULT_LIST_STATUS_NUM_THREADS, INPUT_DIR, INPUT_DIR_RECURSIVE, LIST_STATUS_NUM_THREADS, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method and Description
      List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext job) 

      • Methods inherited from class org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat

        createRecordReader, getFormatMinSplitSize, listStatus

      • Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

        addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, isSplitable, makeSplit, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize

    • Constructor Detail

      • ShuffledSequenceFileInputFormat

        public ShuffledSequenceFileInputFormat()

    • Method Detail

      • getSplits

        public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext job)
                                                       throws IOException
        Overrides:
        getSplits in class org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>
        Throws:
        IOException