本文整理汇总了Java中org.embulk.spi.PageReader类的典型用法代码示例。如果您正苦于以下问题：Java PageReader类的具体用法？Java PageReader怎么用？Java PageReader使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
PageReader类属于org.embulk.spi包，在下文中一共展示了PageReader类的20个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: serialize
import org.embulk.spi.PageReader; //导入依赖的package包/类
@Override
public byte[] serialize(Mode mode, PageReader reader, Schema schema, List<String> dataColumns, TimestampFormatter[] timestampFormatters) {
S3PerRecordOutputColumnVisitor visitor;
switch(mode) {
case SINGLE_COLUMN:
visitor = new MessagePackSingleColumnVisitor(reader, timestampFormatters);
schema.lookupColumn(dataColumns.get(0)).visit(visitor);
break;
case MULTI_COLUMN:
visitor = new MessagePackMultiColumnVisitor(reader, timestampFormatters);
for (String columnName : dataColumns) {
schema.lookupColumn(columnName).visit(visitor);
}
break;
default:
throw new RuntimeException("never reach here");
}
return visitor.getByteArray();
}
开发者ID:tomykaira，项目名称:embulk-output-s3_per_record，代码行数:20，代码来源:S3PerRecordOutputPlugin.java
示例2: testOpenSuccessfully
import org.embulk.spi.PageReader; //导入依赖的package包/类
@Test
public void testOpenSuccessfully()
{
ConfigSource configSource = loadConfigSource("testOpen.yml");
final Schema inputSchema = Schema.builder()
.add("qb", STRING)
.add("qs", STRING)
.add("qa", STRING)
.build();
testQueryString(configSource, inputSchema, "/path?q1=1&q2=2#fragment", new AssertionWithPage()
{
@Override
public void run(PageReader pageReader, TestPageBuilderReader.MockPageOutput pageOutput)
{
for (Page page : pageOutput.pages) {
pageReader.setPage(page);
assertThat(pageReader.getString(0), is("before"));
assertThat(pageReader.getString(1), is("1"));
assertEquals(2L, pageReader.getLong(2));
assertThat(pageReader.getString(3), is("after"));
}
}
});
}
开发者ID:mwed，项目名称:embulk-filter-query_string，代码行数:26，代码来源:TestQueryStringFilterPlugin.java
示例3: testOpenSuccessfullyWithHashbang
import org.embulk.spi.PageReader; //导入依赖的package包/类
@Test
public void testOpenSuccessfullyWithHashbang()
{
ConfigSource configSource = loadConfigSource("testOpen.yml");
final Schema inputSchema = Schema.builder()
.add("qb", STRING)
.add("qs", STRING)
.add("qa", STRING)
.build();
testQueryString(configSource, inputSchema, "/#!/path?q1=one&q2=2#fragment", new AssertionWithPage()
{
@Override
public void run(PageReader pageReader, TestPageBuilderReader.MockPageOutput pageOutput)
{
for (Page page : pageOutput.pages) {
pageReader.setPage(page);
assertThat(pageReader.getString(0), is("before"));
assertThat(pageReader.getString(1), is("one"));
assertEquals(2L, pageReader.getLong(2));
assertThat(pageReader.getString(3), is("after"));
}
}
});
}
开发者ID:mwed，项目名称:embulk-filter-query_string，代码行数:27，代码来源:TestQueryStringFilterPlugin.java
示例4: testQueryString
import org.embulk.spi.PageReader; //导入依赖的package包/类
private void testQueryString(ConfigSource configSource, final Schema inputSchema, final String path, final AssertionWithPage assertion)
{
final QueryStringFilterPlugin plugin = new QueryStringFilterPlugin();
plugin.transaction(configSource, inputSchema, new FilterPlugin.Control()
{
@Override
public void run(TaskSource taskSource, Schema outputSchema)
{
TestPageBuilderReader.MockPageOutput mockPageOutput = new TestPageBuilderReader.MockPageOutput();
PageOutput pageOutput = plugin.open(taskSource, inputSchema, outputSchema, mockPageOutput);
List<Page> pages = PageTestUtils.buildPage(runtime.getBufferAllocator(), inputSchema, "before", path, "after");
for (Page page : pages) {
pageOutput.add(page);
}
pageOutput.finish();
pageOutput.close();
PageReader pageReader = new PageReader(outputSchema);
assertion.run(pageReader, mockPageOutput);
;
}
});
}
开发者ID:mwed，项目名称:embulk-filter-query_string，代码行数:26，代码来源:TestQueryStringFilterPlugin.java
示例5: filter
import org.embulk.spi.PageReader; //导入依赖的package包/类
private List<Object[]> filter(PluginTask task, Schema inputSchema, Object ... objects)
{
TestPageBuilderReader.MockPageOutput output = new TestPageBuilderReader.MockPageOutput();
Schema outputSchema = BuildJsonFilterPlugin.buildOutputSchema(task, inputSchema);
PageBuilder pageBuilder = new PageBuilder(runtime.getBufferAllocator(), outputSchema, output);
PageReader pageReader = new PageReader(inputSchema);
BuildJsonVisitorImpl visitor = new BuildJsonVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);
List<Page> pages = PageTestUtils.buildPage(runtime.getBufferAllocator(), inputSchema, objects);
for (Page page : pages) {
pageReader.setPage(page);
while (pageReader.nextRecord()) {
outputSchema.visitColumns(visitor);
pageBuilder.addRecord();
}
}
pageBuilder.finish();
pageBuilder.close();
return Pages.toObjects(outputSchema, output.pages);
}
开发者ID:hiroyuki-sato，项目名称:embulk-filter-build_json，代码行数:22，代码来源:TestBuildJsonVisitorImpl.java
示例6: ColumnCaster
import org.embulk.spi.PageReader; //导入依赖的package包/类
ColumnCaster(PluginTask task, Schema inputSchema, Schema outputSchema, PageReader pageReader, PageBuilder pageBuilder)
{
this.task = task;
this.inputSchema = inputSchema;
this.outputSchema = outputSchema;
this.pageReader = pageReader;
this.pageBuilder = pageBuilder;
buildTimestampParserMap();
buildTimestampFormatterMap();
buildFromTimestampUnitMap();
buildToTimestampUnitMap();
JsonCaster jsonCaster = new JsonCaster(task, timestampParserMap, timestampFormatterMap, fromTimestampUnitMap, toTimestampUnitMap);
this.jsonVisitor = new JsonVisitor(task, jsonCaster);
}
开发者ID:sonots，项目名称:embulk-filter-timestamp_format，代码行数:17，代码来源:ColumnCaster.java
示例7: Icu4jPageOutput
import org.embulk.spi.PageReader; //导入依赖的package包/类
public Icu4jPageOutput(TaskSource taskSource, Schema inputSchema, Schema outputSchema, PageOutput output) {
this.task = taskSource.loadTask(PluginTask.class);
this.keyNameColumns = Lists.newArrayList();
this.transliteratorsList = Lists.newArrayList();
this.inputSchema = inputSchema;
this.outputSchema = outputSchema;
for (String keyName : task.getKeyNames()) {
keyNameColumns.add(inputSchema.lookupColumn(keyName));
}
for (Map<String, String> setting : task.getSettings()) {
List<Transliterator> tokenizers = Lists.newArrayList();
for (String convertType : setting.get("transliterators").split(",")) {
Transliterator transliterator = Transliterator.getInstance(convertType);
tokenizers.add(transliterator);
}
transliteratorsList.add(tokenizers);
}
reader = new PageReader(inputSchema);
builder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
}
开发者ID:toyama0919，项目名称:embulk-filter-icu4j，代码行数:22，代码来源:Icu4jPageOutput.java
示例8: WootheePageOutput
import org.embulk.spi.PageReader; //导入依赖的package包/类
public WootheePageOutput(TaskSource taskSource, Schema inputSchema, Schema outputSchema, PageOutput output) {
this.task = taskSource.loadTask(PluginTask.class);
this.outputColumns = outputSchema.getColumns();
this.inputColumns = inputSchema.getColumns();
Map<String, Column> inputColumnMap = Maps.newHashMap();
this.wootheeColumnMap = Maps.newHashMap();
for (Column column : outputColumns) {
if (!inputColumns.contains(column)) {
wootheeColumnMap.put(column.getName(), column);
} else {
inputColumnMap.put(column.getName(), column);
}
}
this.reader = new PageReader(inputSchema);
this.builder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
this.keyNameColumn = inputColumnMap.get(task.getKeyName());
}
开发者ID:toyama0919，项目名称:embulk-filter-woothee，代码行数:18，代码来源:WootheePageOutput.java
示例9: filter
import org.embulk.spi.PageReader; //导入依赖的package包/类
private List<Object[]> filter(PluginTask task, Schema inputSchema, Object ... objects)
{
MockPageOutput output = new MockPageOutput();
Schema outputSchema = ColumnFilterPlugin.buildOutputSchema(task, inputSchema);
PageBuilder pageBuilder = new PageBuilder(runtime.getBufferAllocator(), outputSchema, output);
PageReader pageReader = new PageReader(inputSchema);
ColumnVisitorImpl visitor = new ColumnVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);
List<Page> pages = PageTestUtils.buildPage(runtime.getBufferAllocator(), inputSchema, objects);
for (Page page : pages) {
pageReader.setPage(page);
while (pageReader.nextRecord()) {
outputSchema.visitColumns(visitor);
pageBuilder.addRecord();
}
}
pageBuilder.finish();
pageBuilder.close();
return Pages.toObjects(outputSchema, output.pages);
}
开发者ID:sonots，项目名称:embulk-filter-column，代码行数:22，代码来源:TestColumnVisitorImpl.java
示例10: testOpen
import org.embulk.spi.PageReader; //导入依赖的package包/类
@Test
public void testOpen(final @Mocked PageReader reader, final @Mocked PageBuilder builder, final @Mocked Page page) throws Exception {
new NonStrictExpectations() {{
taskSource.loadTask(PluginTask.class); result = task;
task.getDelimiter(); result = "";
reader.nextRecord(); result = true; result = false;
}};
SpeedometerFilterPlugin plugin = new SpeedometerFilterPlugin();
PageOutput output = plugin.open(taskSource, schema, schema, inPageOutput);
output.add(page);
new Verifications() {{
taskSource.loadTask(PluginTask.class); times = 1;
builder.addRecord(); times = 1;
builder.finish(); times = 0;
reader.nextRecord(); times = 2;
reader.setPage(page); times = 1;
schema.visitColumns(withInstanceOf(ColumnVisitor.class)); times = 1;
}};
}
开发者ID:hata，项目名称:embulk-filter-speedometer，代码行数:22，代码来源:TestSpeedometerFilterPlugin.java
示例11: testFinish
import org.embulk.spi.PageReader; //导入依赖的package包/类
@Test
public void testFinish(final @Mocked PageReader reader, final @Mocked PageBuilder builder, final @Mocked Page page) throws Exception {
new NonStrictExpectations() {{
taskSource.loadTask(PluginTask.class); result = task;
task.getDelimiter(); result = "";
}};
SpeedometerFilterPlugin plugin = new SpeedometerFilterPlugin();
PageOutput output = plugin.open(taskSource, schema, schema, inPageOutput);
output.finish();
new Verifications() {{
taskSource.loadTask(PluginTask.class); times = 1;
builder.finish(); times = 1;
}};
}
开发者ID:hata，项目名称:embulk-filter-speedometer，代码行数:17，代码来源:TestSpeedometerFilterPlugin.java
示例12: testClose
import org.embulk.spi.PageReader; //导入依赖的package包/类
@Test
public void testClose(final @Mocked PageReader reader, final @Mocked PageBuilder builder, final @Mocked Page page) throws Exception {
new NonStrictExpectations() {{
taskSource.loadTask(PluginTask.class); result = task;
task.getDelimiter(); result = "";
}};
SpeedometerFilterPlugin plugin = new SpeedometerFilterPlugin();
PageOutput output = plugin.open(taskSource, schema, schema, inPageOutput);
output.close();
new Verifications() {{
taskSource.loadTask(PluginTask.class); times = 1;
builder.close(); times = 1;
reader.close(); times = 1;
}};
}
开发者ID:hata，项目名称:embulk-filter-speedometer，代码行数:18，代码来源:TestSpeedometerFilterPlugin.java
示例13: JoinFilePageOutput
import org.embulk.spi.PageReader; //导入依赖的package包/类
JoinFilePageOutput(
Schema inputSchema,
Schema outputSchema,
Column joinBaseColumn,
HashMap<String, HashMap<String, String>> table,
List<Column> joinColumns,
HashMap<String, TimestampParser> timestampParserMap,
PageOutput pageOutput
)
{
this.pageReader = new PageReader(inputSchema);
this.pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, pageOutput);
this.joinBaseColumn = joinBaseColumn;
this.table = table;
this.joinColumns = joinColumns;
this.timestampParserMap = timestampParserMap;
}
开发者ID:civitaspo，项目名称:embulk-filter-join_file，代码行数:18，代码来源:JoinFilePageOutput.java
示例14: getCurrentJoinBaseColumnValue
import org.embulk.spi.PageReader; //导入依赖的package包/类
private static String getCurrentJoinBaseColumnValue(PageReader pageReader, Column joinBaseColumn)
{
if (pageReader.isNull(joinBaseColumn)) {
return null;
}
if (Types.STRING.equals(joinBaseColumn.getType())) {
return pageReader.getString(joinBaseColumn);
}
else if (Types.BOOLEAN.equals(joinBaseColumn.getType())) {
return String.valueOf(pageReader.getBoolean(joinBaseColumn));
}
else if (Types.DOUBLE.equals(joinBaseColumn.getType())) {
return String.valueOf(pageReader.getDouble(joinBaseColumn));
}
else if (Types.LONG.equals(joinBaseColumn.getType())) {
return String.valueOf(pageReader.getLong(joinBaseColumn));
}
else if (Types.TIMESTAMP.equals(joinBaseColumn.getType())) {
return String.valueOf(pageReader.getTimestamp(joinBaseColumn));
}
throw Throwables.propagate(new Throwable("Unsupported Column Type: " + joinBaseColumn.getType()));
}
开发者ID:civitaspo，项目名称:embulk-filter-join_file，代码行数:25，代码来源:JoinFilePageOutput.java
示例15: GoogleTranslateApiPageOutput
import org.embulk.spi.PageReader; //导入依赖的package包/类
public GoogleTranslateApiPageOutput(TaskSource taskSource, Schema inputSchema, Schema outputSchema, PageOutput output) {
this.task = taskSource.loadTask(PluginTask.class);
this.outputSchema = outputSchema;
this.inputColumns = inputSchema.getColumns();
this.reader = new PageReader(inputSchema);
this.builder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
this.keyNameColumns = Lists.newArrayList();
for (String keyName : task.getKeyNames()) {
this.keyNameColumns.add(outputSchema.lookupColumn(keyName));
}
this.translate = createTranslateService();
this.srcLang = task.getSourceLang().isPresent() ? TranslateOption.sourceLanguage(task.getSourceLang().get()) : null;
this.model = task.getModel().isPresent() ? TranslateOption.model(task.getModel().get()) : null;
}
开发者ID:toyama0919，项目名称:embulk-filter-google_translate_api，代码行数:16，代码来源:GoogleTranslateApiPageOutput.java
示例16: SolrPageOutput
import org.embulk.spi.PageReader; //导入依赖的package包/类
public SolrPageOutput(SolrClient client, Schema schema, PluginTask task) {
this.logger = Exec.getLogger(getClass());
this.client = client;
this.pageReader = new PageReader(schema);
this.schema = schema;
this.task = task;
this.bulkSize = task.getBulkSize();
this.maxRetry = task.getMaxRetry();
}
开发者ID:yahoojapan，项目名称:embulk-output-solr，代码行数:10，代码来源:SolrOutputPlugin.java
示例17: buildKey
import org.embulk.spi.PageReader; //导入依赖的package包/类
private String buildKey(@NotNull PageReader pageReader) {
StringBuilder sb = new StringBuilder();
for (KeyPart p : keyPattern) {
sb.append(p.resolve(pageReader));
}
return sb.toString();
}
开发者ID:tomykaira，项目名称:embulk-output-s3_per_record，代码行数:8，代码来源:S3PerRecordOutputPlugin.java
示例18: BuildJsonVisitorImpl
import org.embulk.spi.PageReader; //导入依赖的package包/类
BuildJsonVisitorImpl(BuildJsonFilterPlugin.PluginTask task, Schema inputSchema, Schema outputSchema, PageReader pageReader, PageBuilder pageBuilder)
{
this.task = task;
this.inputSchema = inputSchema;
this.outputSchema = outputSchema;
this.pageReader = pageReader;
this.pageBuilder = pageBuilder;
initializeJSONParseTree();
}
开发者ID:hiroyuki-sato，项目名称:embulk-filter-build_json，代码行数:10，代码来源:BuildJsonVisitorImpl.java
示例19: open
import org.embulk.spi.PageReader; //导入依赖的package包/类
@Override
public PageOutput open(TaskSource taskSource, final Schema inputSchema,
final Schema outputSchema, final PageOutput output)
{
final PluginTask task = taskSource.loadTask(PluginTask.class);
return new PageOutput()
{
private PageReader pageReader = new PageReader(inputSchema);
private PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
private BuildJsonVisitorImpl visitor = new BuildJsonVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);
@Override
public void finish()
{
pageBuilder.finish();
}
@Override
public void close()
{
pageBuilder.close();
}
@Override
public void add(Page page)
{
pageReader.setPage(page);
while (pageReader.nextRecord()) {
outputSchema.visitColumns(visitor);
pageBuilder.addRecord();
}
}
};
}
开发者ID:hiroyuki-sato，项目名称:embulk-filter-build_json，代码行数:37，代码来源:BuildJsonFilterPlugin.java
示例20: ColumnCaster
import org.embulk.spi.PageReader; //导入依赖的package包/类
ColumnCaster(TypecastFilterPlugin.PluginTask task, Schema inputSchema, Schema outputSchema,
PageReader pageReader, PageBuilder pageBuilder)
{
this.task = task;
this.inputSchema = inputSchema;
this.outputSchema = outputSchema;
this.pageReader = pageReader;
this.pageBuilder = pageBuilder;
buildTimestampParserMap();
buildTimestampFormatterMap();
this.jsonVisitor = new JsonVisitor(task, inputSchema, outputSchema);
}
开发者ID:sonots，项目名称:embulk-filter-typecast，代码行数:14，代码来源:ColumnCaster.java
注：本文中的org.embulk.spi.PageReader类示例整理自Github/MSDocs等源码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。 |
客服电话
APP下载
官方微信
请发表评论