* copyright (C) 1984-2019 merrill consultants dallas texas usa

MXG design, to first reconstruct legitimate VB records

without truncated sections, and then process those SMF

records in a separate process. I get to do this because

IBM regards this as "working as documented".

-But, fortunately, JMF records are not typically created

every day; they tend to be turned on and then off for a

study, so we can live with waiting a day to process all

of the JMF records from a study. And, this one very

small glitch in just one part of one subtype of the VERY

comprehensive JES 3 Monitoring Facility is now bypassed,

so all the rest of the JMF datasets are fully usable,

all of the other segments of this subtype 1 record, and

even those first 78 FCT entries are output; MXGWARN notes

on the SAS log alert you there were FCT segments missed.
-The specific case investigated: There were more FCT than

would fit in one SMF record. FCT entries are variable

length sections with many optional subsections. JMF

created 4 physical SMF records, segment numbers 1-4:

-The First segment has JMF PROD, JMF GENE, R84JES3O and

R84IRBSC offsets populated and has no FCT entries.

-The Second segment doesn't have JMF PROD nor JMF GENE

(needed for Interval Start and End times), and has only

R84FCTO, which points to a valid FCT "Header" section

(but R84FAWLN is the TOTAL length of all FCTs, and in

R84FCTN=250 is the total count of all FCTs in all segs;

what's needed here is how many FCTs are in THIS record!

This second segment then contains 78 complete FCT entry

sections, with R84FSEQN 1 thru 78, but then, only the

first part of FCT number 79 was written in this SEGNR=2

SMF record: the last 72 bytes were truncated.

-The Third segment also has no JMF PROD nor GENE sections

and does NOT contain an FCT Header section; it contains

only FCT entries, but it's data area STARTS with those

last 72 bytes of FCT entry number 79, and then FCT entry

number 80 thru 187 are contained in this SEGNR=3 record.

-Finally, the Fourth Segmented Record is like the third,

no PROD/GENE/FCT Header, only FCT entries, but its data

area (happens?) to start with FCT Entry R84FSEQN=188's

byte one, and then FCTs 189 thru 250 are complete in the

SEGNR=4 JMF record.

-Unrelated to the above issues, truncated subtype 2

records were discovered; IBM APAR OA37982 is now open to

correct that error.

Thanks to Lucy F. Trabulus, Bank of America, USA.

Change 29.211 Support for Throughput Manager APAR TMT6214, corrects the

VMACTPMX invalid subtype 16 triplets when there were more than 76

Sep 21, 2011 steps in a job. Those invalid triplets caused the ERROR:

INPUT STATEMENT EXCEEDED RECORD. The APAR corrects the

error by creating multiple "continuation" records when

when there are more than 76 steps, and installing their

APAR eliminates the INPUT STATEMENT EXCEEDED ERROR.

A new version of MXG is NOT required to support the APAR.

But, to create observations in TPM16J dataset, you need

Change 29.200, in MXG 29.06, to correct an unrelated MXG

coding error that caused zero observations to be created.

And, job-info variables were only added to the TPM16J

dataset by THIS change, which will be in MXG 29.07, or

can be requested now via email from [e-mail]).

-Support for Subtype 5 PCS segment is added, with those

variables added to the existing TPMSLM dataset when the

PCS segment is present. The variable names are taken

verbatim from the DSECT, so most are longer than the old

8-byte ASM DSECT field names.
Change 29.210 The spelling of most variables available in this header

IHDRTMVS exit for selection were wrong. And, variable LMRKJOBC is

VMACTMVS NOT the JOB name of the Landmark Monitor for z/OS, but is

Sep 14, 2011 the SYSTEM ID of that system, so it's label is corrected,

and it can be used to select by SYSTEM.

Thanks to Sam Bass, McLane Co., USA.

Change 29.209 DB2 V10 ID=100 SUBTYPE=1 with INVALID HEADER caused INPUT

VMACDB2H EXCEEDED RECORD LENGTH error. The defective record had

Sep 14, 2011 0122x (290 decimal) for the offset to the un-truncated

QWHSOPID field, with that QWHSTYP=2 starting in byte 5375

but 5375+290=5665, while the LENGTH of the data portion

was only 5534 bytes. Protection for this invalid header

field has been added for all of the "untruncated" fields

in the QWHSTYP=2 Header segment. IBM might be contacted.

Thanks to Tom Pierce, LabCorp, USA.
Change 29.208 IBM i-Series (AS400) with more than 32 CPUs create two

VMACQACS records in QAPMSYSC dataset (IBM "QAPMSYSCPU") for each

Sep 14, 2011 interval, with the first 32 engines in the first record

and the second 32 engines in the second record. You must

now invoke the _SQAPSYC macro after your _TQAPSYC macro

when you read the QAPMSYSC infile:

%INCLUDE SOURCLIB(VMACQACS);

_TQAPCON

_TQAPSYC

_SQAPSYC

as the _SQAPSYC macro now sums those two records to

create a single observation per interval, with the

(corrected) PCTCPUBY percentage, and the new variable

SCPUSUM with the total CPU time for all online engines.

Thanks to Karen Florup, Bank of America, USA.
Change 29.207 MONWRITE Broken Control Record error due to incomplete

VMACVMXA decoding of the 2.14 (SCLALL) Schedule Domain record.

Sep 13, 2011 The error message gives no clue the cause is the 2.14.

This is the first MONWRITE file in a long time that had

SCL (Schedule) domain records enabled; they are rarely

turned on due to large volume, and provide low-level

trace-like event data that requires pretty deep knowledge

of z/VM internals to be useful, but now they too are

supported.

Thanks to David J. Schumann, Blue Cross of Minnesota, USA.

Change 29.206 Cosmetic. VMXGSUM now dies gracefully with an MXGERROR:

FORMATS message if there is no input dataset to process.

VMACEDGR

Sep 13, 2011

Thanks to
Change 29.205 Cosmetic. New format MGEDGCF decodes variable COMEFROM

FORMATS to identify which variable will be missing when invalid

VMACEDGR dates (e.g., 2000/000) are found in RMM records.

Sep 13, 2011

Thanks to
====== Changes thru 29.204 were in MXG 29.06 dated Sep 8, 2011========
Change 29.204 -Support for RMF III RCD data, promised in Change 29.187,

EXZRBRCX is now completed. The new ZRBRCDX dataset contains the

IMACRMFV Response Times for Reporting Classes and the existing

VMACRMFV ZRBRCDT datasets is updated with Response Times for the

VMXGINIT Service Classes.

Sep 7, 2011 -BY lists BZRTRCS/RCR/RCP/RCD were revised.

Sep 13, 2011 -Cosmetic changes, PCT, Version, etc.
Change 29.203 The labels for variables PC24RHWM and PC24SHWM for the

VMAC110 "Below 16MB" areas had ERDSA and ESDSA but those labels

Sep 7, 2011 should have been RDSA and SDSA.

Thanks to Steven Kimball, Aetna, USA.

Thanks to Victoria Lepak, Aetna, USA.
Change 29.202 The _IDEDGSA macro default value should be 512 but it was

VMACEDGS left with 8Ax (138) after a test with user's data, and if

Sep 7, 2011 you have an ID=138 record in your SMF file, JCLTEXTx will

fail (INPUT STATEMENT EXCEEDED).

Thanks to Walter Noniewicz, State of Connecticut, USA.
Change 29.201 -Support for DB2 V10 restructured IFCID=217 record, which

VMAC102 caused T102T217 and T102U217 to have zero observations,

Sep 7, 2011 but (fortunately) the incompatible record format did NOT

cause an execution error. Dataset T102S217 now has only

one unique variable, and new QW0217AL and QW0217AS ASIDs

were added to T102T217 & T102U217 datasets respectively.

Dataset T102V217 always has zero observations (after V8).

-Unrelated, many offseg+lenseg LE LENGTH segment tests are

corrected to offseg+lenseg-1; the incorrect test could

have prevented reading legitimate segments.

Thanks to Betty Wong, Bank of America, UDS.
Change 29.200 No observations were created in TPM16J dataset because

VMACTPMX the MXG test comparing segment offset+length with the

Sep 7, 2011 record length was off by one, preventing input/output.

Thanks to Scott Barry, SBBWorks Inc, USA.

Change 29.199 The %SYSPROD(GRAPH) function doesn't validate that the

GOPTIONS product is installed; it only confirms that the SETINIT

DOC licenses SAS/GRAPH. The MXG test in VMXGINIT to set

VMXGINIT GOPTIONS for SAS/GRAPH examples in MXG GRAFxxxx members

Sep 4, 2011 (that was added without a change number in MXG 28.05)

is true when the SETINIT said yes, but GOPTIONS/PATTERNn

statements fail when SAS/GRAPH isn't actually present,

with a 180 syntax error on the GOPTIONS statement, since

those procedure-specific statements can't compiled if the

product isn't installed. The new PROC PRODUCT_STATUS will

list all products that are installed, but its use would

require a PROC PRINTTO to trap its output to a file, a

DATA step to read that file, and new %macro variable and

logic, just to set these (optional!) SAS/GRAPH options.

So, to prevent a 180 error in VMXGINIT, these statements

GOPTIONS COLORS=(R B G CYAN MAGENTA ORANGE GREY BROWN BLACK);

PATTERN1 V=S; PATTERN2 V=L1; PATTERN3 V=R1; PATTERN4 V=X1;

PATTERN5 V=L2; PATTERN6 V=R2; PATTERN7 V=X2; PATTERN8 V=L3;

PATTERN9 V=R3; PATTERN10 V=X3;

are no longer set in VMXGINIT, but are moved into the new

GOPTIONS member, and all GRAFxxxx members were updated to

set them with %INCLUDE SOURCLIB(GOPTIONS);

-Also, MXG macro variables SASGRAF and SASSTAT are removed

from VMXGINIT; they were never referenced in any MXG code

and were set by %SYSPROD(), so they too could be wrong.

Thanks to Richard Haynes, Blue Cross Blue Shield of Kansas, USA.

====== Changes thru 29.198 were in MXG 29.06 dated Sep 1, 2011========
Change 29.198 iSERIES variable GDES21 was multiplied by 4096, but it

VMACQACS should have been multiplied by 1024; when GDES11 is all

Aug 31, 2011 nines, GDES21 contains the storage assigned to the LPAR.

Thanks to David Bixler, FISERV, USA.

Change 29.197 Two new segments in Velocity Software XAM Version 4.1 CPU

VMACXAM record are decoded (PRCINS,PRCDIA) but not output as they

Aug 31, 2011 are repeated and hence need new datasets created, and the

new STOASC (DEV) and new SYTLCK (SYS) weren't documented;

these four segments were protected for the NEW SEGMENT

messages but were not decoded. See Change 29.253.

Change 29.196 MXG calculation of GMTOFF30 in SMF 30 records could be

VMAC30 off by 0.01 second, only for positive GMT Offsets, but

Aug 31, 2011 that impacted the BUILDPDB logic that combines MULTIDD=Y

records for a step, when there were intervening SMF 30s

from a different job in between the MULTIDD=Y records.

The MULTIDD=Y records don't contain SMF30IST, so GMTOFF30

must be retained from the prior non-MULTIDD record, but

in this case, SMF30IST was 0.07 with SMF30ISS 0.073983 in

the non-MULTIDD, but an intervening step's non-record had

SMF30ISS of 0.075983 with SMF30IST of 0.08, causing MXG's

(failing) fuzzy logic use GMTOFF30 of .01 in MULTIDD=Ys

from the first step. This caused BUILDPDB to not include

the EXCP/IOTM counts in PDB.SMFINTRV for the MULTIDD's,

so while TYPE30_V had 26 million EXCPs, only 24 million

were in PDB.SMFINTRV for that day's DB2 intervals.

The real culprits, aside from MXG's failing fuzzy logic,

is the absence of GMTOFF in SMF 30 records, the absence

of SMF30IST in the MULTIDD=Y records, and SMF30ISS being

an 8-byte (LOCAL) TODSTAMP with microsecond resolution,

while SMF30IST (GMT) is an 8-byte SMFSTAMP with only 10

millisecond resolution. The correction for MXG's logic

error is to first use ROUND(SMF30ISS,.01) before its

comparison with SMF30IST so both have 10 millisecond,

rounded, values, so GMTOFF30 is correctly calculated for

both positive and negative GMT offsets.

A formal request to IBM SMF30 development to add the long

overdue GMT Offsset has again been made so that MXG logic

is not needed to compensate for its absence.

Thanks to Giuseppe Giacomodonato, EPVTECH, ITALY.
Change 29.195 MXG now terminates reading TMS data with USER ABEND 1234

VMACTMS5 and an ERROR message if the input file is not the TMC.

Aug 30, 2011 The TMSGRW TMS utility program creates 340 byte records

that MXG read, but they produced incorrect results.

MXG support requires the TMC data as its input file.

Thanks to Srini Anne, State of California, USA.

Change 29.194 Statistics on Page Datasets are added to RMFINTRV (so if

VMXGRMFI your DB2 DBAs add massive buffers and eat up all of your

Aug 27, 2011 page datasets, you'll at least be able to prove why the

system died for lack of paging space!). New variables:

MAXCOMN ='MAXIMUM*COMMON*SLOTS USED'

MAXLOCL ='MAXIMUM*LOCAL*SLOTS USED'

MAXPLPA ='MAXIMUM*PLPA*SLOTS USED'

NRDSLOCL='ACTIVE*LOCAL*DATASETS'

PCTLOCL ='AVERAGE*PERCENT*SLOTS USED*ALL LOCALS'

PCTMAXLO='MAXIMUM*PERCENT*SLOTS USED*ANY LOCAL'

PCTCOMN ='PERCENT*COMMON*SLOTS USED'

PCTPLPA ='PERCENT*PLPA*SLOTS USED'

SIO75CNT='PAGING*SIO*COUNT'

SLOTCOMN='COMMON*SLOTS*DEFINED'

SLOTLOCL='LOCAL*SLOTS*DEFINED'

SLOTPLPA='PLPA*SLOTS*DEFINED'

Thanks to Chuck Hopf, Independent Consultant, USA.
Change 29.193 -ANALZPCR with PDB=SMF reads 113s as well as the others

ANALZPCR and invokes ASUM113.

UTILBLDP -UTILBLDP adds SMF 113 processing, conditionally:

Aug 27, 2011 If BUILDPDB EQ YES and USERADD does not contain 113 it is

added to USERADD, and if INCLAFTR doesn't contain ASUM113

it is added there.

Thanks to Chuck Hopf, Independent Consultant, USA.
Change 29.192 If the length of text in the list of DDNAMEs passed into

UTILCONT UTILCONT exceeded 256 bytes, the list was truncated and

Aug 26, 2011 remaining datasets were not read. Increased to 32000.

Thanks to David Bixler, FISERV, USA.

Change 29.191 Windows 7 may error when attempting to write files to the

DOC root directory (e.g., C:\ if that is your boot directory)

Aug 24, 2011 or to the c:\windows or the c:\program files directories,

with a message about "Insufficient authorization to use".

The error may be circumvented if the program is "run as

authorized", but some MXG examples of ASCII syntax had

"c:\filename.xxx" that could cause the error, so all

instances of that syntax were revised to explicitly

have a directory name (e.g. "c:\mxg\filename.xxx').

In addition, some ASCII LIBNAME examples (which always

point to a directory rather than a file) did not have

a final slash character (LIBNAME PDB 'C:\PDB';) and thus

were unclear that the syntax requires a directory name.

Those examples are now changed to include a final slash

(LIBNAME PDB 'c:\pdb\';) to reinforce that the object is

a directory and not a filename.

Change 29.190 In this "JCLSPLIT" example, CICSTRAN was incorrectly sent

BLDSPSMA to //WORK DD instead of the correct //CICSTRAN DDname.

JCLSPSMA

Aug 23, 2011

Thanks to Art Cuneo, Blue Cross Blue Shield of Illinois, USA.
Change 29.189 Support for Software AG's ADABAS SMF User record creates

EXADUSER sixteen new datasets:

EXADPARM

EXADSTG DDDDDD MXG MXG

EXADIODD DATASET DATASET DATASET

EXADTHRD SUFFIX NAME LABEL

EXADFILE

EXADCMD ADUSER ADABUSER ADABAS USER RECORD

EXADCSP ADPARM ADABPARM ADABAS PARM RECORD

EXADCSG ADSTG ADABSTG ADABAS STG RECORD

EXADCSB ADIODD ADABIODD ADABAS IODD RECORD

EXADCSF ADTHRD ADABTHRD ADABAS THRD RECORD

EXADLOK ADFILE ADABFILE ADABAS FILE RECORD

EXADMSGB ADCMD ADABCMD ADABAS CMD RECORD

EXADMSGC ADCSP ADABCSP ADABAS CSP RECORD

EXADMSGH ADCSG ADABCSG ADABAS CSG RECORD

EXADREV ADCSB ADABCSB ADABAS CSB RECORD

IMACADAB ADCSF ADABCSF ADABAS CSF RECORD

TYPEADAB ADLOK ADABLOK ADABAS LOK RECORD

TYPSADAB ADMSGB ADABMSGB ADABAS MSGB RECORD

VMACADAB ADMSGC ADABMSGC ADABAS MSGC RECORD

VMXGINIT ADMSGH ADABMSGH ADABAS MSGH RECORD

Aug 24, 2011 ADREV ADABREV ADABAS REV RECORD

-Datasets ADPARM, ADSTG, ADTHRD, ADFILE, and ADCMD are now

decoded and validated with SMF records; the others have

been coded but are untested with actual data records.

-Datasets ADABTHRD and ADABFILE do not have fields that

identify the Thread/File; counters ADTHRDNR and ADFILENR

are created as the sequence number in each record, but in

the test data, only the first instance in each record has

a non-zero count (but there are many instances in each

record with zero counts that perhaps should be deleted).

Thanks to Wayne Campos, Corelogic, USA.
Change 29.188 -Blank TRANNAME in ASUMUOW occurred with MXG 28.05 that

IMACUOW were not present with MXG 28.04, so new Case 4 example is

VMXGUOW created to handle this sequence of DB2/CICS/MQ records.

Aug 23, 2011 Precipitated possibly by our changes to VMXGUOW, as the

only site difference is that remote program definitions

with a remote tranid on them are used.

-New debugging TRACEUOW=TRAN option was added/needed to

diagnose this problem: at the endpoint, it tells you

everything there is to know about that particular UOW

when the TRANNAME comes up blank.

-All of the macros that are typically tailored are now

defined in the IMACUOW member, and you should EDIT that

member into your "USERID.SOURCLIB(IMACUOW) and make your

tailoring there, and should NOT now ever have VMXGUOW in

your "USERID.SOURCLIB" tailoring library.

Thanks to Dave Campbell, SunTrust, USA.

Change 29.187 -MXG 29.05 only. ASMRMFV could fail with 0C4 ABEND. Under

ASMRMFV some conditions, IBM did not create a Period Data

VMACRMFV Extension for a Report Class in the RCD record. The count

Aug 23, 2011 field was unexpectedly zero, causing a loop counter to

Sep 1, 2011 become negative, and the 0C4 during an MVCL command.

This condition was never observed during testing.

-This ASMRMFV properly outputs the RCD records, but the

MXG code in VMACRMFV has not yet been updated to read

those records, and prior VMACRMFV could fail trying to

read the new RCD output, so this VMACRMFV skips over any

RCDG3 records. If you want those data, contact support

since the VMACRMFV updates should be done by the time you

read this text.

Thanks to Neil Ervin, Wells Fargo Bank, USA.

Thanks to Jason, Bierman, Great Lakes Higher Education Corp., USA.
Change 29.186 Support for InfoSphere Classic Federation Server Account

EXCFSACC user SMF record creates two new datasets:

EXCFSACV

IMACCFS

FORMATS DDDDDD MXG MXG

TYPECFS DATASET DATASET DATASET

TYPSCFS SUFFIX NAME LABEL

VMACCFS

VMXGINIT CFSACC CFSACCT INFOSPHERE CLASSIC FEDSRVR CPU

Aug 15, 2011 CFSACV CFSVIOL INFOSPHERE CLASSIC FEDSRVR VIOLATE

These possible violations are formatted in MGCFSTY:

101:NOT AUTH TO ISSUE A DROP TABLE FOR TABLE

102:NOT AUTH TO ISSUE A DROP INDEX FOR TABLE

103:NOT AUTH TO ISSUE A DROP VIEW FOR VIEW

104:NOT AUTH TO ISSUE A DROP PROCEDURE FOR STORED PROC

200:NOT AUTH TO CREATE TABLE

201:NOT AUTH TO CREATE INDEX

202:NOT AUTH TO ISSUE THE CREATE VIEW STATEMENT

203:NOT AUTH TO ISSUE THE CREATE PROCEDURE STATEMENT

300:NOT AUTH TO ISSUE A SELECT STATEMENT FOR TABLE/VIEW

301:NOT AUTH TO ISSUE AN UPDATE STATEMENT FOR TABLE

302:NOT AUTH TO ISSUE AN INSERT STATEMENT FOR TABLE

303:NOT AUTH TO CALL STORED PROCEDURE

403:NOT AUTH TO ISSUE A DELETE STATEMENT AGAINST TABLE

500:NOT AUTH TO ISSUE GRANT STATEMENT

501:NOT AUTH TO ISSUE REVOKE STATEMENT

Thanks to Jeana M. Bechtel, Edward D. Jones, USA.

Thanks to Jim Poletti, Edward D. Jones, USA.

Change 29.185 GRAFWRKX used the old IMACWORK workloads to graph the

GRAFWRKX workload variables and had work1-work99 parameters

Aug 13, 2011 for the new. This change removes the need to specify

workload parameters unless you want to alter the order

in which the bars are stacked. Support for the old

IMACWORK workloads is removed but VGETWKLD is used to

capture all of the workloads unless you specify WORK1-

WORK99 values. If you do not specify WORKx values the

workloads are stacked alphabetically from bottom of the

bar to the top. Except that uncaptured will ALWAYS be

at the bottom. If you specify WORK1-x workloads then

the bars are stacked bottom to top starting with WORK1

and going up numerically. You can still use the old

IMACWORK definitions but the parameters are dead and

do not function.
Change 29.184 GRAFRMFI used the old IMACWORK workloads to graph the

GRAFRMFI workload variables and had no provision for the new

VGETWKLD VMXGRMFI sets of workload variables. It now uses

Aug 13, 2011 VGETWKLD to find the workloads that are defined in

your RMFINTRV dataset (whichever method is used) and

uses those workloads and labels to graph the workload

variables.
This is part of the ODS change 29.169, but the change in

the member are so extensive that it warranted a separate

change.

Numerous issues corrected in this change/circumvention:

-TIME VALUE NOT VALID FOR TIME FORMAT. SAS V9.3 only.

if all obs had a missing value for a TIME variable, that

WARNING was printed; circumvented with WHERE clause, but

that raised another WARNING about WHERE CLAUSE changing.

The more insidious problem was the structure:

PROC GPLOT;

PLOT A

PLOT B

PLOT C

PLOT D

PLOT E ....

Which worked fine unless all the obs had missing values

for one of the variables. For example, if A has all

missing values, then, instead of skipping over B, it

made a copy of A. Even worse, IF B C D E all had

missing values, then you got 5 copies of A instead of

the 1 you really wanted.

The solution was to change the structure to:

PROC GPLOT;

PLOT A

PROC GPLOT;

PLOT B

PROC GPLOT;

PLOT C

PROC GPLOT;

PLOT D

PROC GPLOT;

PLOT E ....

It runs a little slower but it eliminated that problem

as well as the warning about the where clause changing.

WARNING: This routine can produce hundreds of graphs;

the number is a function of the number of system, days,

and workloads.

-Code to execute from any populated RMFINTRV dataset:

%GRAFRMFI(PDB=WORK,

ODSTYPE=HTML,

ODSPATH=c:\HTMLTEST,

ODSFILE=GRAFRMFIBODY.HTML);

Change 29.183 Variable SMF19VLI never existed and is no longer created.

VMAC19 This is an overlay for SMF19TRK and SMF19TRM which were