本文整理汇总了Java中org.apache.tez.runtime.api.Reader类的典型用法代码示例。如果您正苦于以下问题：Java Reader类的具体用法？Java Reader怎么用？Java Reader使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
Reader类属于org.apache.tez.runtime.api包，在下文中一共展示了Reader类的14个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: attachInputs
import org.apache.tez.runtime.api.Reader; //导入依赖的package包/类
@Override
public void attachInputs(Map<String, LogicalInput> inputs,
Configuration conf)
throws ExecException {
this.conf = conf;
LogicalInput input = inputs.get(inputKey);
if (input == null) {
throw new ExecException("Input from vertex " + inputKey + " is missing");
}
try {
Reader r = input.getReader();
if (r instanceof KeyValueReader) {
reader = (KeyValueReader) r;
} else {
shuffleInput = true;
shuffleReader = (KeyValuesReader) r;
hasNext = shuffleReader.next();
}
LOG.info("Attached input from vertex " + inputKey + " : input=" + input + ", reader=" + r);
} catch (Exception e) {
throw new ExecException(e);
}
}
开发者ID:sigmoidanalytics，项目名称:spork，代码行数:25，代码来源:POValueInputTez.java
示例2: attachInputs
import org.apache.tez.runtime.api.Reader; //导入依赖的package包/类
@Override
public void attachInputs(Map<String, LogicalInput> inputs,
Configuration conf) throws ExecException {
LogicalInput input = inputs.get(inputKey);
if (input == null) {
throw new ExecException("Input from vertex " + inputKey + " is missing");
}
try {
Reader r = input.getReader();
if (r instanceof KeyValueReader) {
reader = (KeyValueReader) r;
} else {
shuffleInput = true;
shuffleReader = (KeyValuesReader) r;
}
LOG.info("Attached input from vertex " + inputKey + " : input=" + input + ", reader=" + r);
} catch (Exception e) {
throw new ExecException(e);
}
}
开发者ID:sigmoidanalytics，项目名称:spork，代码行数:21，代码来源:POIdentityInOutTez.java
示例3: run
import org.apache.tez.runtime.api.Reader; //导入依赖的package包/类
@Override
public void run() throws Exception {
Preconditions.checkState(getInputs().size() == 1);
Preconditions.checkState(getOutputs().size() == 1);
LogicalInput input = getInputs().values().iterator().next();
Reader rawReader = input.getReader();
Preconditions.checkState(rawReader instanceof KeyValueReader);
LogicalOutput output = getOutputs().values().iterator().next();
KeyValueReader reader = (KeyValueReader) rawReader;
KeyValueWriter writer = (KeyValueWriter) output.getWriter();
while (reader.next()) {
Object val = reader.getCurrentValue();
writer.write(val, NullWritable.get());
}
}
开发者ID:apache，项目名称:incubator-tez，代码行数:18，代码来源:IntersectExample.java
示例4: run
import org.apache.tez.runtime.api.Reader; //导入依赖的package包/类
@Override
public void run() throws Exception {
Preconditions.checkState(getInputs().size() == 2);
Preconditions.checkState(getOutputs().size() == 1);
// Get the input data for the 2 sides of the join from the 2 inputs
LogicalInput logicalInput1 = getInputs().get(input1);
LogicalInput logicalInput2 = getInputs().get(input2);
Reader inputReader1 = logicalInput1.getReader();
Reader inputReader2 = logicalInput2.getReader();
Preconditions.checkState(inputReader1 instanceof KeyValuesReader);
Preconditions.checkState(inputReader2 instanceof KeyValuesReader);
LogicalOutput lo = getOutputs().get(joinOutput);
Preconditions.checkState(lo.getWriter() instanceof KeyValueWriter);
KeyValueWriter writer = (KeyValueWriter) lo.getWriter();
join((KeyValuesReader) inputReader1, (KeyValuesReader) inputReader2,
writer);
}
开发者ID:apache，项目名称:tez，代码行数:19，代码来源:SortMergeJoinExample.java
示例5: run
import org.apache.tez.runtime.api.Reader; //导入依赖的package包/类
@Override
public void run() throws Exception {
Preconditions.checkState(getInputs().size() == 1);
Preconditions.checkState(getOutputs().size() == 1);
// not looking up inputs and outputs by name because there is just one
// instance and this processor is used in many different DAGs with
// different names for inputs and outputs
LogicalInput input = getInputs().values().iterator().next();
Reader rawReader = input.getReader();
Preconditions.checkState(rawReader instanceof KeyValueReader);
LogicalOutput output = getOutputs().values().iterator().next();
KeyValueReader reader = (KeyValueReader) rawReader;
KeyValueWriter writer = (KeyValueWriter) output.getWriter();
while (reader.next()) {
Object val = reader.getCurrentValue();
// The data value itself is the join key. Simply write it out as the
// key.
// The output value is null.
writer.write(val, NullWritable.get());
}
}
开发者ID:apache，项目名称:tez，代码行数:24，代码来源:HashJoinExample.java
示例6: run
import org.apache.tez.runtime.api.Reader; //导入依赖的package包/类
@Override
public void run() throws Exception {
Preconditions.checkState(getInputs().size() == 2);
Preconditions.checkState(getOutputs().size() == 0);
LogicalInput lhsInput = getInputs().get(LHS_INPUT_NAME);
LogicalInput rhsInput = getInputs().get(RHS_INPUT_NAME);
Reader lhsReaderRaw = lhsInput.getReader();
Reader rhsReaderRaw = rhsInput.getReader();
Preconditions.checkState(lhsReaderRaw instanceof KeyValuesReader);
Preconditions.checkState(rhsReaderRaw instanceof KeyValuesReader);
KeyValuesReader lhsReader = (KeyValuesReader) lhsReaderRaw;
KeyValuesReader rhsReader = (KeyValuesReader) rhsReaderRaw;
TezCounter lhsMissingKeyCounter = getContext().getCounters().findCounter(COUNTER_GROUP_NAME,
MISSING_KEY_COUNTER_NAME);
while (lhsReader.next()) {
if (rhsReader.next()) {
if (!lhsReader.getCurrentKey().equals(rhsReader.getCurrentKey())) {
LOG.info("MismatchedKeys: " + "lhs=" + lhsReader.getCurrentKey() + ", rhs=" + rhsReader.getCurrentKey());
lhsMissingKeyCounter.increment(1);
}
} else {
lhsMissingKeyCounter.increment(1);
LOG.info("ExtraKey in lhs: " + lhsReader.getClass());
break;
}
}
if (rhsReader.next()) {
lhsMissingKeyCounter.increment(1);
LOG.info("ExtraKey in rhs: " + lhsReader.getClass());
}
}
开发者ID:apache，项目名称:incubator-tez，代码行数:34，代码来源:IntersectValidate.java
示例7: run
import org.apache.tez.runtime.api.Reader; //导入依赖的package包/类
@Override
public void run() throws Exception {
Preconditions.checkState(getInputs().size() == 2);
Preconditions.checkState(getOutputs().size() == 0);
LogicalInput lhsInput = getInputs().get(LHS_INPUT_NAME);
LogicalInput rhsInput = getInputs().get(RHS_INPUT_NAME);
Reader lhsReaderRaw = lhsInput.getReader();
Reader rhsReaderRaw = rhsInput.getReader();
Preconditions.checkState(lhsReaderRaw instanceof KeyValuesReader);
Preconditions.checkState(rhsReaderRaw instanceof KeyValuesReader);
KeyValuesReader lhsReader = (KeyValuesReader) lhsReaderRaw;
KeyValuesReader rhsReader = (KeyValuesReader) rhsReaderRaw;
boolean rhsReaderEnd = false;
TezCounter lhsMissingKeyCounter = getContext().getCounters().findCounter(COUNTER_GROUP_NAME,
MISSING_KEY_COUNTER_NAME);
while (lhsReader.next()) {
if (rhsReader.next()) {
if (!lhsReader.getCurrentKey().equals(rhsReader.getCurrentKey())) {
LOG.info("MismatchedKeys: " + "lhs=" + lhsReader.getCurrentKey() + ", rhs=" + rhsReader.getCurrentKey());
lhsMissingKeyCounter.increment(1);
}
} else {
lhsMissingKeyCounter.increment(1);
LOG.info("ExtraKey in lhs: " + lhsReader.getClass());
rhsReaderEnd = true;
break;
}
}
if (!rhsReaderEnd && rhsReader.next()) {
lhsMissingKeyCounter.increment(1);
LOG.info("ExtraKey in rhs: " + rhsReader.getClass());
}
}
开发者ID:apache，项目名称:tez，代码行数:36，代码来源:JoinValidate.java
示例8: getReader
import org.apache.tez.runtime.api.Reader; //导入依赖的package包/类
@Override
public Reader getReader() throws Exception {
return new KeyValueReader() {
String[] keys = new String[numRecordPerSrc];
int i = -1;
@Override
public boolean next() throws IOException {
if (i == -1) {
for (int j = 0; j < numRecordPerSrc; j++) {
keys[j] = ""+j;
}
}
i++;
return i < keys.length;
}
@Override
public Object getCurrentKey() throws IOException {
return keys[i];
}
@Override
public Object getCurrentValue() throws IOException {
return keys[i];
}
};
}
开发者ID:apache，项目名称:tez，代码行数:30，代码来源:CartesianProduct.java
示例9: getReader
import org.apache.tez.runtime.api.Reader; //导入依赖的package包/类
@Override
public Reader getReader() throws Exception {
if (this.exLocation == ExceptionLocation.INPUT_HANDLE_EVENTS) {
synchronized (condition) {
// wait for exception thrown from handleEvents. Otherwise,
// processor may exit before the exception from handleEvents is
// caught.
condition.wait();
}
}
if (this.exLocation == ExceptionLocation.INPUT_GET_READER) {
throw new Exception(this.exLocation.name());
}
return null;
}
开发者ID:apache，项目名称:tez，代码行数:16，代码来源:TestExceptionPropagation.java
示例10: getReader
import org.apache.tez.runtime.api.Reader; //导入依赖的package包/类
@Override
public Reader getReader() throws Exception {
return null;
}
开发者ID:apache，项目名称:incubator-tez，代码行数:5，代码来源:TestInput.java
示例11: getReader
import org.apache.tez.runtime.api.Reader; //导入依赖的package包/类
@Override
public Reader getReader() throws Exception {
return new ConcatenatedMergedKeyValueReader();
}
开发者ID:apache，项目名称:incubator-tez，代码行数:5，代码来源:ConcatenatedMergedKeyValueInput.java
示例12: getReader
import org.apache.tez.runtime.api.Reader; //导入依赖的package包/类
@Override
public Reader getReader() throws Exception {
return new ConcatenatedMergedKeyValuesReader();
}
开发者ID:apache，项目名称:incubator-tez，代码行数:5，代码来源:ConcatenatedMergedKeyValuesInput.java
示例13: getReader
import org.apache.tez.runtime.api.Reader; //导入依赖的package包/类
@Override
public Reader getReader() throws Exception {
throw new UnsupportedOperationException("getReader not supported. use getKeyValueReaders");
}
开发者ID:apache，项目名称:incubator-tez，代码行数:5，代码来源:MultiMRInput.java
示例14: getReader
import org.apache.tez.runtime.api.Reader; //导入依赖的package包/类
@Override
public Reader getReader() throws Exception {
return reader;
}
开发者ID:apache，项目名称:tez，代码行数:5，代码来源:TestSortedGroupedMergedInput.java
注：本文中的org.apache.tez.runtime.api.Reader类示例整理自Github/MSDocs等源码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。 |
客服电话
APP下载
官方微信
请发表评论