* copyright (C) 1984-2019 merrill consultants dallas texas usa

the same CEC. While SYNCTIME is constant in all RMF

records for a "normal" interval, when a WLM Policy

Change was made for six of nine LPARs in a CEC, there

were six 1.5-minute-DURATM obs, three 15-minute-DURATM

obs, and then six 13.5-minute obs. Those six 1.5

minute short duration records, written as a result of

the policy change, each had a unique SYNCTIME, so it

could not be used to group the observations. Only the

SMF70GIE value appears to be safe to use to get all of

the pieces together in CEC and 70PR/70LP summarization

and this redesign now uses SMF70GIE in place of the

STARTIME for the grouping into summary intervals.

(RMF Development later confirmed that they also use

SMF70GIE for their report grouping. 26Apr2006.)

5. PDB.ASUMCEC will contain invalid data if your LPARs

are on different time zones; you must use TIMEBILD to

force all of the datetimes to a single time zone for

ASUMCEC/ASUMCELP to be valid.

6. For ASUMCEC/ASUMCELP, comparisons should be extremely

close, but possibly not exact, for "normal" intervals.

For intervals were some LPARs were not present for the

full interval, or when a Policy Change was issued,

i.e., when there are more than one STARTIME in an

SMF70GIE interval, the new code should be correct,

creating only one observation for each SMF70GIE,

whereas the old code had an observation for each

STARTIME, and the old data for those "broken"

intervals was usually incorrect.

7. The circumvention member EXCECTIM is now unnecessary

and is no longer referenced by MXG code.

Thanks to Alain Delaroche, CA-Atlantica, FRANCE.
Change 23.321 Support for Split RMF 70 records: CRITICAL. SPLIT70.

VMAC7072 -If you have enough LPARs (32) with SPARE engines, IBM now

XMAC7072 splits the RMF 70 data into multiple physical SMF records

TEST70SP that are completely INCOMPATIBLE and REQUIRE this change.

Dec 7, 2005 MXG KNEW SPLIT RECORDS COULD EXIST AND PRINTS TEN ERROR

Jan 19, 2006 MESSAGES ON THE LOG THAT THERE ARE ERRORS AND BAD DATA.

Change 22.344, Jan, 2005, added that logic in MXG 22.22.

-But until I had actual records, I couldn't revise MXG.

-The support required a complete restructure of the code

and logic that creates the TYPE70 and TYPE70PR datasets

from RMF or CMF type 70 SMF records. None of the other

datasets created in the VMAC7072 "product" code were

touched by this change. Work started at CMG, and across

the holidays in chunks of development and testing using

RMF and CMF data from 33 sites that were not split (so

the old and new logic could be compared; the old logic

created trashed data with split records so there is no

"old" for split). Instead, the ANALRMFR RMF-like report

was compared with IBM RMF reports from the one split RMF

70 record that precipitated this major redesign!

-As long as you use BUILDPDB/3, RMFINTRV, TYPE7072 or the

TYPS7072 MXG code to read your RMF/CMF 70/72 records, the

MXG redesign will execute transparently with MXG 23.10+:

- Dataset WORK.TYPE70 initially has zero observations

after the SMF data is read; instead, new WORK.TYPE70SP

dataset will have observations, whether your records

are split or not.

- The WORK.TYPE70PR dataset is created from SMF, but it

is completely invalid if it was created from a SPLIT

SMF 70 record.

- The redesign now must invoke the _STY70 Data Set Sort

macro, which processes the WORK.TYPE70SP/TYPE70PR data

and in that process, creates these temporary datasets:

SORT70SP - Sort of TYPE70SP

PHYSICAL - LPARNAME='PHYSICAL' obs from TYPE70PR

NUCPCXPU - Count CPs, ICFs, IFLs, IAFs from PHYSICAL

THISPART - PARTISHN=LPARNUM subset from PHYSICAL

FROM70PR - Summary, create LPARn vars from THISPART

Three (SORT70SP, FROM70PR, NUCPSCPU) are then merged

to create the PDB.TYPE70 output dataset, which is also

enhanced by the addition of these new variables:

LPARNAME SMF70BDA LCPUDEDn-x, LCPUWAIn-x

Then _STY70 logic sorts TYPE70PR into SORT70PR so it

can be MERGEd with NUCPSCPU to populate the PARTNCPU

and NRICFCPU/NRIFLCPU/NRIFACPU variables from the

PHYSICAL segments to create the PDB.TYPE70PR dataset.

-If you use YOUR OWN CODE that uses the _VAR7072 _CDE7072

macros, then you MUST also invoke the _STY70 macro to

create the TYPE70 and TYPE70PR datasets; otherwise, you

will have zero observations in TYPE70 and bad data in the

TYPE70PR dataset. (If you already use the _S7072 product

sort macro to sort all of the 7072 datasets, you're home

free, since the product sort macro always invokes the

data set sort macro for all datasets from that product.)

The default output DD for _STY70 is "PDB" for both the

TYPE70 and TYPE70PR datasets. If you want your old

code to continue to write those two datasets to the

"WORK" DDname, you would insert:

%LET PTY70=WORK;

%LET PTY70PR=WORK;

before your DATA step.

-Note that _STY70PR sort macro is now null and must be

replaced with _STY70.

-The MXG TYPE7072 member now must invoke the _STY70 macro,

but with %LETs inserted in TYPE7072 so that the rebuilt

TYPE70/TYPE70PR datasets are still written to //WORK.

(So an existing %INCLUDE SOURCLIB(TYPE7072) will still

create those datasets in the //WORK library.)

But if you use %INCLUDE SOURLIB(TYPE7072), you can not

add an invocation of _STY70, because of that under the

covers execution. And you'll get an error if you do:

-You can get ERROR: FILE WORK.TYPE70SP DOES NOT EXIST:

a. if you do attempt to invoke _STY70 a second time,

b. if you have an old VMAC7072 in your "USERID" tailoring

source libraries

c. if your predecessor had (incorrectly) tailored MXG and

had redefined MACRO _VAR7072 or MACRO _CDE7072 macros

(usually in IMACKEEP member). It has ALWAYS been

incorrect installation tailoring to redefine those

critical MACRO _VARxxxx or MACRO _CDExxxx macros in

your USERID MXG Tailoring Library, precisely because

they block MXG from it correct definitions.

-ITRM/ITSV section: revised Jan 31, 2006:

Most ITRM sites do NOT have to do anything at all for the

new 70's architecture, as long as you create XRMFINT (the

RMFINTRV MXG dataset) with your %CMPROCES ITRM execution.

ONLY if you process RMF 70 records, but do NOT create

the RMFINTRV dataset, you will, at present, need to

insert the text _STY70 in your EXPDBOUT member.

This original change text said all ITRM sites needed

to add that statement, but that text was wrong, and it

is the rare site that would process 70s without also

creating the XRMFINT/RMFINTRV dataset. A later change

to ITRM can be expected that will make the invocation

of _STY70 internal to ITRM, and that statement will

need to be removed at that time.

-This iteration has no protection for incomplete sets of

split records: first record filled today's SMF file, next

record is in tomorrow's SMF file. That may be done in a

later iteration, using the //SPIN DD to hold data.

-VSETZERO member has been removed from the VMAC70s logic.

It was added in MXG 23.03 by Change 23.070 in an

attempt to deal with inconsistent STARTIME values

across LPARs, by setting STARTIME to the exact minute

when seconds were 58,59, or 01-10.

It is removed because the unchanged STARTIME value must

be used for the merge of the split records, but also

because VMXG70PR is revised in Change 23.323 to use the

SMF70GIE variable for the merge, and the minimum value of

STARTIME is used only for reporting those summary data.

-In both rewrites, I discovered a few cases in which the

old MXG code was inconsistent or even wrong:

- LPARn variables for unused LCPUADDRs were occasionally

zeros when they should have been a missing value; now

they are always missing. Had no impact on your data,

but they cause false positives in PROC COMPARE in the

TEST70SP program because they are different.

- Handling of engines with CAI='00'x (not on at end of

interval) and CAI='02'x or CAI='03'x (error flag on)

was inconsistent in the old code. Since SMF70ONT now

should be populated, it is used to determine if each

engine's dispatch/wait times are included in totals,

impacting the values in those individual "n" engine

variables and the totals, in particular variables

MVSWAITn,CPUWAITn,CPUPDTMn,CPUEFFMn and

MVSWAITM,CPUWAITM,CPUPDTMM,CPUEFFMM

If these values differ in your compare, look first at

the CAIn variables to see if they account for the

compare differences, knowing the new is correct now.

- Because values are now summed and then divided that

were previously divided and then summed, some small

differences (less than .001 absolute value) were found

in some time and percentage variables, but they only

impacted the PROC COMPARE, and not the real world.

- Duplicate SMF records were not always deleted, which

caused PCTCPUBY in ASUM70PR and ASUMCEC over 100%.

The sort order of TYPE70PR to remove duplicates is:

BY SYSPLEX SYSTEM SYNCTIME STARTIME SMF70GIE LCPUADDR;

- Variables NRICFCPU and NRIFAS in ASUM70PR/ASUMCEC were

sometimes incorrect in the old code.

- The SMF6CNF bit macro no longer exists; not needed with

the revised handling of the CAI '02'x and '03'x values.

- APAR OA14024 just reported that the IBM RMF Report Writer

fails with an ABEND 0C4 when it tried to read a split SMF

70 record from a system with 60 LPARs, because the total

length of all records was then over 64K bytes, a value

that killed the report writer, due use of a signed field.

And this happened after an RMF Developer told me that it

was very simple to reassemble their split data!

- This SMF 70 record really did not need to be split by

IBM now! The record contained 368 LCPUADDR segments from

the 32 LPARs, but there were actually only 68 LCPUADDR in

use; all of the other LCPUADDR segments were for each of

the SPARE engines on the z9, so if IBM had chosen to only

write the non-SPARE engine's, this rewrite would not have

occurred now. But, the rewrite is now done so the future

is safe.

- New TEST70SP member creates TYPE70/TYPE70PR datasets with

the old MXG 23.09 code (in XMAC7072 member) and with the

new VMAC7072 code, and uses PROC COMPARE to identify any

differences between new and old logic. TEST70SP can ONLY

be used with NON-SPLIT RMF 70s.

(The old code didn't support split records, which cause

multiple TYPE70 observations with same STARTIME and

with incorrect data values.)

It also created new and old ASUM70PR and ASUMCEC data and

compares them, but if your data has multiple STARTIMEs in

an SMF70GIE interval, the old and new will not compare,

as the old was split into multiple observations, but the

new logic correctly summarizes those multiples, so there

will be fewer observations in the new compared with the

old, which PROC COMPARE reports as lots of differences,

but there is no error.

Thanks to Matt Clarke, Charles Schwab & Co., Inc, USA.

Thanks to Neil Ervin, Charles Schwab & Co., Inc, USA.
Change 23.320 -Truncated SMF 102 record caused INPUT STATEMENT EXCEEDED

VMAC102 RECORD LENGTH error is now detected and reported on the

Dec 23, 2005 SAS log as a truncated record, rather than an ABEND.

-IFCID 83 record from DB2 V7.1 caused similar ABEND, but

this was because the MXG code for that IFCID was only

for DB2 V8.1, as only that version's test data had been

received; now, both V7 and V8 IFCID 83 are supported.

Thanks to Tony Pratt, Tampa Electric, USA.

Change 23.319 New ASUMDB2 program summarizes DB2ACCT events to create

ASUMDB2 interval (hourly default) transaction counts, resources,

VMXGINIT and response time buckets, using the same macro variables

Dec 18, 2005 to define those buckets as are used in ASUMCICS/ASUMCICX

that creates PDB.CICS.

Thanks to John Rivera, (i)Structure, USA.

Change 23.318 Incorrect syntax with character text for NRPERIODS caused

VMXGRMFI strange and unobvious error messages; now, if NRPERIODS

Dec 18, 2005 is not a number, the execution will ABEND so you can fix

your syntax.

Thanks to Robbie A. McCoy, Salt River Project, USA.
Change 23.317 -Syntax error in TRNDUOW due to a semi-colon after a ELSE.

TRNDCEC -Variable NRPRCS NOT FOUND in TRNDCEC because it was

TRNDUOW removed some time ago.

Dec 18, 2005 -TRND members had not been fully tested in QA, now are.

Thanks to Freddie Arie, Merrill Consultants, USA.
Change 23.316 Changes to VGETENG required corresponding changes to this

UTILCONT utility that provides the size of contents of your SAS

Dec 15, 2005 data libraries.

Thanks to Paul Naddeo, FISERV, USA.

Change 23.315 Dataset ICFD70PR NOT SORTED error due to short interval

ASUM70PR RMF records required yet another sort to force proper

Dec 15, 2005 sequence.

Thanks to Jimmy J. Hu, Ameren, USA.

Change 23.314 Documentation only. APAR OA06476 documents that TYPE74CA

VMAC74 fields R745INCR, R745BYTR, R745BYTW, R745RTIR, R745RTIW

Dec 13, 2005 have never been populated and are now reserved fields

that will always be zeros. The data was planned for the

2105 800 models, but the data was not available on those

boxes. The data has become available with the 2105 box,

but that interface and APAR OA06476 adds new R7451INC,

and R7451CT1-4 to RAID Rank/Extent Pool Section.

Change 23.313 -Variables R791TIFC/R792TIFC were incorrectly normalized

VMAC79 by R791NFFI/R792NFFI, but they are already in CP seconds

Dec 12, 2005 so they are no longer normalized.

-Variables R791TCP and R729TCP are now deaccumulated, and

the label for R792TCP is now correct.

-Text in Change 22.152 now has correct variable names and

labels for TYPE791 and TYPE792 IFA-related variables.

-Change 23.132 updated to list the TYPE79 changes made.

Thanks to Rick Southby, Insurance Australia Group (IAG), AUSTRALIA.
Change 23.312 New CICSBAD dataset contains "bad" CICSTRAN observations,

EXCICBAD that were previously output in CICSTRAN dataset, but are

IMACCICS known to be defective, and so are now output in CICSBAD.

VMAC110 There are two "bad" conditions supported in this change:

VMXGINIT -When a user types in a TRANNAME that is not a valid name,

Dec 9, 2005 there still is a type 110 segment created with missing

Feb 9, 2006 fields, and TRANNAME contains whatever was typed. Those

records are identified by PROGRAM='########', and are

now output into dataset CICSBAD rather than CICSTRAN.

While these records are usually just typos, hackers have

been known to keep typing names until they find one that

works, so these records might be useful in CICS security

issues. Each record has between 3 and 4 milliseconds of

CPU time in TASCPUTM, which I presume is the fixed cost

to create a CICSTRAN observation (speculation - there may

be uncaptured CPU) on a SU_SEC=10000 speed engine.

-When a CICS task is executing on an OPEN TCB, and is then

purged, APAR PQ86971 documents that all of the clocks are

invalid when this happens, and that APAR adds a new bit

to the TRANFLAG field to identify these tasks. Since the

clock values are all wrong, these transactions are now

also output in CICSBAD instead of CICSTRAN.

-So, CICSBAD will now contain all PROGRAM='########' obs,

and all transactions purged on an open TCB, and CICSTRAN

will no longer have any of those invalid transactions.

-The format for CPU-time fields were changed to TIME13.3,

to display the millisecond level; TIME16.6 would show the

full resolution, but would cause reports based on the

current length to be shifted, so it was not chosen.
Change 23.311 Support for Beta 91 Version 3.1.1.0, which INCOMPATIBLY

VMACBE91 changed the header of their record, and all subtypes.

Dec 8, 2005 This iteration supports BETA91B and BETA91C datasets,

Dec 12, 2005 which were validated with '44'x and '50'x subtypes. Two

other datasets have not been tested with real records.

Thanks to Herve Brusseaus, Fortis Bank, LUXEMBOURG.

Change 23.310 Variable ESMFNTRN was always missing because it existed

VMACMPLX only in a LABEL statement, and was misspelled. The count

Dec 8, 2005 of transactions is in variable ESMFTRAN.

Thanks to Sudie Wulfert-Schickedanz, Anheuser-Busch., USA.

====== Changes thru 23.309 were in MXG 23.09 dated Dec 7, 2005=========
Change 23.309 INVALID ARGUMENT TO FUNCTION SQRT error when an invalid

VMXGUOW transaction (PROGRAM='########') was processed. In the

TRNDUOW VMXGUOW member, CICSTRAN obs with that PROGRAM name are

Dec 7, 2005 now deleted prior to summary, and TRNDUOW is protected.

Thanks to Bruce Widlund, Merrill Consultants, USA.
Change 23.308 MXG 23.08-23.09. Note VARIABLE JOBINFO1 IS UNINITIALIZED

VMAC26J2 because of an incomplete ending-comment; fortunately, the

Dec 5, 2005 only impact was that these four variables: OPTJOURN,

OPTOUTPT, TYPRUN and RESTARTD will all be blank.

Thanks to Jon Whitcomb, Great Lakes Educational Loan Service, USA.
Change 23.307 INPUT STATEMENT EXCEEDED after "DTP=24 SEGMENT.." message

VMAC80A because the additional segments were not processed after

Dec 5, 2005 that MXG message.

Dec 9, 2005 -Dec 9. Same error corrected for DTP=25 SEGMENT.

Thanks to Coen Wessels, Unicible, SWITZERLAND.

Thanks to Mark Nouwen, Dexia, BELGIUM.

Change 23.306 MXG variables PARTNCPU and NRICFCPU/NRIFACPU/NRIFLCPUs

VMAC7072 still wrong on z9s, because IBM's SMF70BNP field is in

VMXG70PR error on z9s. MXG calculated PARTNCPU from SMF70BNP,

Dec 5, 2005 but in earlier platforms, SMF70BNP included the count of

Dec 7, 2005 ICFs, which MXG subtracted. So when the z9 added new

'IFA' and 'IFL' values to SMF70CIN, I assumed those also

needed to be subtracted from SMF70BNP. Now, however,

IBM confirmed SMF70BNP is in error and will be changed

with the next z9 microcode:

the expected MCL number containing this change is

J99671.014 in driver d63j (Jan, 2006)

and recommended that instead to count of engines in the

PHYSICAL segments, rather than use SMF70BNP, so this

change does set PARTNCPU from the count of CPs in the

PHYSICAL partition records.

Dec 7: The Dec 5 change accidentally fixed the z9 data

and un-fixed other processors, because I had

spelled PARTNCPU as PLATCPUS in both the code

and text of the original change. PARTNCPU is

the name in the type 70 datasets; PLATCPUS is

the name only in the RMFINTRV dataset.

-In VMXG70PR, NRICFCPU and NRIFACPU could also be wrong;

a SUM was used where a MAX was required; because my test

data has only one of each, the SUM and MAX were the same

and the logic error was not detected.

-Variable NRPRCS is no longer kept in PDB.ASUM70PR and

PDB.ASUMCEC datasets; it has no meaning for those data.

Thanks to Dave Crandall, Farmers Insurance, USA.

Change 23.305 Optional OPID variable was never INPUTed!

IMACICO1

Dec 2, 2005

Thanks to Lucy Fukushima, California Health and Welfare, USA

Change 23.304 ANALCNCR could fail when an observation with missing

ANALCNCR values for the number of concurrent events was being

Dec 2, 2005 generated, which caused the calculation of the Y-axis

to fail. The record with missing values is dropped and

axix calculations were protected for the possibility of

missing values.

Thanks to Bruce Hewson, Citibank N.A., SINGAPORE.
====== Changes thru 23.303 were in MXG 23.10 dated Dec 1, 2005=========
Change 23.303 First MXG 23.09 only, and only under SAS Version 8: ERROR

VGETENG LIBNAMES IS NOT A RECOGNIZED PROC SQL DICTIONARY TABLE.

Dec 1, 2005 Change DICTIONARY.LIBNAMES to DICTIONARY.MEMBERS and

Dec 2, 2005 the PROC SQL will work under either V8 or V9.

The LIBNAMES dataset in the DICTIONARY was new in V9,

and we failed to test under V8 with the first 23.09.

Dec 2: We now test version, and use LIBNAMES with V9 or

use MEMBERS with V8. The advantage of the LIBNAMES is

that we can get the engine as long as the LIBNAME has

been referenced (either with a LIBNAME statement, or

implicitly by referencing a dataset), whereas the MEMBERS

only exists if a dataset in the library has been opened.

Thanks to Robbie A. McCoy, Salt River Project, USA.
Change 23.302 -The RACF Unload Utility output have always had a 4-digit

EXRAC2A0 numeric "RECNR", in EBCDIC, but now a new '02A0' value

IMACRACF is found, which caused INVALID DATA FOR RECNR error.

VMACRACF Since numeric RECNR was a kept variable in all datasets,

VMXGINIT new character variable RECTYPE is now INPUT and used in

Nov 30, 2005 all tests, were also changed to character syntax.

-The new RACF02A0 dataset, for USER OVM DATA, is now

created by this change.

-I am now aware of additional new RECTYPE values and they

are listed in the comments in VMACRACF as "NOT DECODED",

as I need test records to support them.

Thanks to Bruce Hewson, Citigroup N.A., SINGAPORE.

Change 23.301 The MXSMFTME and MNSMFTME variables now exclude SMF 2 & 3

VMACSMF header/trailer records (created in the output SMF file

Nov 30, 2005 when IFASMFDP dumps, or later copies, an SMF file). The

ID 2 and 3s record when the dump/copy program ran, and

do not describe the range of timestamps of the SMF data.

Thanks to Erik Janssen, ING Netherlands, THE NETHERLANDS.

====== Changes thru 23.300 were in MXG 23.09 dated Nov 30, 2005=========
Change 23.300 -ML-37 of MXGTMNT monitor now captures SYSLOG messages in

ASMTAPEE a new SMF subtype 8 for these mount-related events:

ASUMTAPE IEC501A IEC501E IEC705I IEF233A IEF502E IEF234E IOS070E

EXTMNSTS IECTMS6 IECTMS9 IOS690I IEF235D

EXTMNSYM and a new subtype 9 record can be written for any other

EXTMNSYS SYSLOG messages you want captured in your SMF data!

IMACTMNT -Two new MXG datasets are created from MXGTMNT's subtypes:

VMACTMNT dddddd dataset description

VMXGINIT TMNSYT TYPESYMT Mount-specific SYSLOG, subtype 8

Nov 28, 2005 TMNSYL TYPESYSL User-selected SYSLOG messages, st 9.

Jun 15, 2006 -Added Jun 15, 2006:

This redesign REQUIRES you to use ASMTAPEE at ML-38 or

later, as it depends on the SYSLOG Mount-specific records

and there will be zero observations in PDB.ASUMTAPE if

there are zero obs in TYPESYMT syslog mount dataset.