Hp Neoview Release 2.3 Software Uživatelský manuál

HP Neoview Transporter User GuideHP Part Number: 545787-001Published: April 2008Edition: Release 2.3

10

Error Message and BehaviorThe control file generator can encounter connection, SQL, and file I/O type errors. If a connectionor a file I/O error is ge

C Global Options TableGlobal Transporter options that are specified in the Options section of the control file can beoverridden at other levels or sec

102

D Reserved KeywordsTransporter Reserved KeywordsAll control file options and section identifiers are reserved in a control file.All keywords are case-

Table D-1 Reserved Keywords List (continued)Reserved Keywordsurlparallelstreamsexpressionusenullstringpasswordextractusernamepipefaileddatafileversion

glossarycontrol file Input to Transporter. The control file specifies how you want to move data from source totarget.Control FileGeneratorA standalone

106

IndexBbaddatafile, description of, 47byteintegral data type, 45byteintintegral data type, 45Cchar[acter]character data type, 45clienton Linux, 58on wi

date/time data type, 45datetimedate/time data type, 45decimalintegral data type, 45deleteonerror, description of, 48delimited data, 63, 64, 65discards

startseq, description of, 53streamsparallel, 21supported data types, 45system, description of, 54Ttablestaging, 21tenacity, description of, 54timedate

About This DocumentThis manual describes how to install and manage HP Neoview Transporter. This product providesprocesses and commands for loading dat

Information about processes and commands for loading data into yourNeoview platform or extracting data from it.Neoview Transporter User GuideInformati

Italic LettersItalic letters, regardless of font, indicate variable items that you supply. Items not enclosedin brackets are required. For example:fil

ATTRIBUTE[S] attribute [, attribute]...{, sql-expression}...An ellipsis immediately following a single syntax item indicates that you can repeat thats

1 OverviewThe HP Neoview Transporter client provides a high-speed load and extract facility to move datato (load) and from (extract) the HP Neoview pl

• “Control File Generator” (page 99)• “Control File Editor ” (page 97)For detailed information see:“Control File Organization and Syntax” (page 25)“Co

2 Installation and ConfigurationThis chapter addresses the following topics:• “Installation Requirements” (page 17)• “Installing Transporter” (page 17

Table 2-1 Transporter Client InstallationActionScreenClick Next to continue installation.1WelcomeRead and accept the Transporter license agreement.2Li

Security Parameters SavedAs part of Transporter client installation, the security parameters are saved in$NVTHOME/conf/security/prop.file . For more i

© Copyright 2008 Hewlett-Packard Development Company, L.P.Legal NoticeConfidential computer software. Valid license from HP required for possession, u

20

3 Functions and FeaturesIn addition to standard load and extract, Transporter provides support for these additionalfeatures and functions:• “Staging T

PrehashingTransporter streamlines the process for load operations with hash-partitioned tables by performinga pre-hashing operation on the client. For

4 SecurityBecause load and extract jobs connect to databases and access password-protected resources, itis necessary for Transporter to keep track of

NOTE: If the environment variable NVTHOME does not exist, nvtencrsrv returns an error.Table 4-2 Options Supported by the nvtencrsrv UtilityDescription

5 Control File Organization and SyntaxThe control file is a text file that tells Transporter how you want your data moved from sourceto target for eit

Table 5-1 Control File OrganizationOptionsControl File SectionN.A.“Version” (page 27)Global options for all Transporter jobs controlled by this file.•

Table 5-1 Control File Organization (continued)OptionsControl File SectionLocation of sources and targets for data.These global options can be overrid

it rejects the control file and reports an error. There cannot be more than 99 minor versionsto a major version.There will be no incompatible changes

• “sorted” (page 53)• “startseq” (page 53)• “system” (page 54)• “tenacity” (page 54)• “timeout” (page 54)• “truncate” (page 54)• “url” (page 54)• “use

Table of ContentsAbout This Document...11Intended

The default is decimal.• Floating Point data types— general — equivalent to printf "%f" format (fixedpoint)— exponential — equivalent to pri

— space— tab— comma (,)— semicolon (;)— slash (/)— backslash (\\)Period (.) is the only supported separator between integral and fractional seconds.Th

“%Y”— To specify a formatstring for a day-time duration of 30.5 seconds formatted as“30.5”:“%S”— To specify a formatstring for a day-time duration of

Dataformats: Delimited Formatformatname delimited[options format-options][fields delimited by character][records separated by character][optionally qu

options format-options“crossoveryear” (page 47)record-length numberSpecifies the length of a data record in unit of characters.For example, each recor

fieldnameThe name you select for the field of the format. If the format is SQL table, the fieldnamemust be one of the column names of the associated S

Only the simple data type name should be used. For example, if a table column namedFIRSTNAME is of data type varchar(8), you should specify “FIRSTNAME

targetformatThe target dataformat as defined in “Dataformats” (page 32).For a load operation, targetformat must be an SQL table dataformat.For an extr

Table 5-3 Datatype Mappings (continued)Can be Mapped to a Target Field of Type...Source Field of Type...any character typedatetimestampdateany charact

discussion of the condition option below for more detail. This option is notsupported for extract operations.• system — the Neoview system generates t

Options...28T

EMP_ID = identity loader• Increments the source field VALUE_SOURCE by 10 and assigns the resulting value to thetarget field VALUE_TARGET:VALUE_TARGET

• Delimited identifiers are case-sensitive.• Spaces within a delimited identifier are significant except for trailing spaces.• You can use reserved wo

Options for pipe sources:• “endseq” (page 48)• “forcestaging” (page 49)• “multipart” (page 50)• “retries” (page 53)• “sorted” (page 53)• “startseq” (p

The Java client accepts control files containing multiple jobs.jobs{job-definition [{job-definition ] ...}job-definition: jobname { load | extract }[o

job-entry-listone or more occurrences of job-entry, separated by a comma (,). The last job-entryshould not use the comma separator.This specifies a li

IncludeYou can include one or more control file fragments from other control files. A control file fragmentis a control file without a version number.

46

6 Control File Optionsbaddatafilebaddatafile = filenameThe default value is current-working-directory/badrecord.dat.For a load operation, this option

Specifies the data source name for ODBC and JDBC connections to the Neoview system.datasource_name must be a server-side data source defined on the Ne

This is an optional feature to allow data synchronization between Transporter and yourapplications.The start and end sequence records are considered “

Requirements...61Paral

This option can be specified at the global level or for individual file or pipe sources that are partof a load operation. When the “parallel” (page 52

.NNNNWhere NNNN is a zero-padded four digit integer.For example, if the specified prefix is “extractData” and the value of parallelstreamsis 4, the fo

an empty string. For all other types, a field containing only white space is rejected and therecord is written to the “baddatafile” (page 47) file.For

password specifies the password to use when connecting to a JDBC, ODBC, or JMS data source.• password specifies a plain-text password.• ${referencenam

For a load operation when a pipe is read, Transporter discards any records read until thestartseq record is seen. Transporter stops reading records on

• JDBC connections to the Neoview system• JMS sourcesThere is no default value.The Transporter Java client uses this URL for its internal database con

56

7 Load and Extract OperationsThe Transporter Client performs load and extract operations based on the content of a controlfile. This chapter addresses

Starting the Client on Linux SystemsUse this shell command to start the Transporter Client. You specify the control file dependingon where the file re

• A source for the data, named in the “Sources” (page 40) section of the control file• A target for the data, also named in the “Sources” (page 40) se

C Global Options Table...101D Reserved Keywords...

CAUTION: If multipart = false, and parallelstreams is enabled, Transporter stillmoves data off the Neoview platform in multiple streams. However, all

RequirementsThese are the requirements to use the Trickle Feed feature:• ActiveMQ (JMS provider), which is included with the Java client package• A se

• The set of targets (that you have set in the “Sources” (page 40) section) must be the samefor all sources in the job.• If multiple job entries speci

8 Data ProcessingThis chapter addresses these topics:“Data For Load Operations” (page 63)“Processing Data for Load Operations” (page 63)“Delimited Dat

Fixed—Width Data for Load OperationsTransporter addresses and processes a fixed—width input field as follows, in this order:1. If “nullstring” (page 5

For Windows platforms: CR/LF (carriage return/line-feed)—— For Unix-like systems: LF (line feed)— For the Macintosh family: CR (carriage return)Quote

optionally qualified by 'Considerations for Character Data• For Load Operations:— Character fields can include any characters valid for the chara

9 TroubleshootingThis section covers job recovery and solutions to problems you might encounter.• “Logs” (page 67)• “Job Recovery” (page 67)• “Job Sta

Considerations for Job RecoveryTransporter jobs can be recovered, with these considerations:• Only load jobs with file sources can be recovered. Job r

• Static— Extract source: Neoview SQL Text or Multi-Partition Set Table Extract— Single Stream or Parallel Extract Streams— Target information: name o

List of Tables1-1 Transporter Client Software Components...162-1 Tra

Example 9-4 Identify All Job Entries that Successfully Completed Within the Last Seven DaysSELECT B.FILE_HOST, B.FILE_ABS_PATH, B.FILE_MOD

Example 9-6 Identify All Jobs Executed From Host abc Using the Latest Version of Control File/home/control files/ControlFile.txtSELECT DISTINCT A.JOB_

72

A Control File ExamplesControl File ExamplesFollowing are control file examples for:• Data types Example A-1 (page 74)• Extract from a Neoview SQL sou

Example A-1 Control File: Datatypes/* All Data Types */version 1;options { errors = 1000, discards = 500, rowsetsize = 1000}typeformats {# c

intv10 interval format intv_frmt_10, intv11 interval format intv_frmt_11, intv12 interval format intv_frmt_12, flot1 fl

tims1 = tims1, tims2 = tims2, intv00 = intv00, intv02 = intv02, intv03 = intv03, intv05 = intv05,

Example A-2 Control File: Extract From Neoview SQL Source/*----------------------------------------*//* Simple Extract From Neoview SQL Source To File

LAST_NAME = LAST_NAME, ADDRESS = ADDRESS, ZIP = ZIP, PHONE = PHONE, INFO1 = INFO1, INFO2

Example A-3 Control File: Extract From Table Source to Named Pipe/*------------------------------------------------*//* Simple Extract From Table Sour

8

{ extract1 extract options ( parallel = "false" ) ( source src1 target tgt1 map map1 )}80 Control File Examples

Example A-4 Control File: Load Fixed Width Data/* Load Fixed Width Data */version 1;options { errors = 1000, discards = 100, rowsetsize

LAST_NAME = LAST_NAME, ADDRESS = ADDRESS, ZIP = ZIP, PHONE = PHONE, SSN = SSN, INFO1

Example A-5 Control File: Include Defining DataformatsYou can use an Include file such as this to define data formats for many control files. SeeExamp

Example A-6 Control File: Include — Load From File to Table/* Simple Load From File To Table Using INCLUDE To Include Control File Fragment (File 1/2)

Example A-7 Control File: Load and Extract/* Simple Load From File To Table & Extract From Neoview SQL Source To File */version 1.0;options{ er

PHONE = PHONE, SSN = SSN, INFO1 = INFO1, INFO2 = INFO2 )map2 from dataformat2 to dataformat3 (

Example A-8 Control File: Multiple Dataformats and Maps/*-----------------------------*//* Multiple Dataformats & Maps *//*-----------------------

( system = "asl0101", user = "user", password = "transport", url

Example A-9 Control File: Reflexive Update/*-------------------------*//* Simple Reflexive Update *//*-------------------------*/version 1.0;options {

List of Examples9-1 Sample HP_TRANSPORTER_BASE_JOB_STATS File...689-2 Sample HP_TRANSPO

Example A-10 Control File: Update with Constant, NULL, or Expression/*--------------------------------*//* Constants, NULLs & Expressions *//*----

load1 load options ( parallel= "false" ) ( source src1 target tgt1 map map1 )}91

Example A-11 Control File: Identity columnThis example of defines an IDENTITY column. CUSTOMER_NUMBER can be any numeric datatype (SMALLINT, INTEGER,

Example A-12 Control File: Usage of NULLSTRING/*--------------------------------*//* Usage of NULLSTRING */ /*-----------------------------

Example A-13 Control File: NOAUDIT mode/*---------------------------------------------------------------------------------------*//* A Single Load Job

Example A-14 Control File: Forcestaging Load with Multiple Job Entries in a Single Job/*--------------------------------------------------------------

source src2 target tgt1 map map1 source src3 target tgt1 map map1 )}96 Control File Examples

B Control File Editor and Control File GeneratorThis chapter addresses the following topics:• The “Control File Editor ” (page 97)• The “Control File

Adding Content to a Control FileClick the appropriate tab to select the section of the control file you want to create or change. AnInput Pane and Ent

Click Insert —> Include or click the Ctrl+I.Then select the control file that you want to insert into the current control file.Control File Generat