by merging the Allocate with Dismount events, so that the
ownership, durations, plus bytes read/written is known!
This major enhancement should have been written long ago.
A "tape event" is created in PDB.ASUMTAPE for every tape
volume dismounted. An event starts with the allocation
for a volume and ends with the dismount of that volume.
An event could be an allocate/mount/dismount with one
open/close of one dataset on that volume, or a tape event
could have opened many datasets on a volume during one
mount of the volume, or an event could allocate a volume
and mount it in one step and then retain the volume and
use it many times in many subsequent steps until dismount
of the volume. All these are recognized and handled.
These are important variables in the new PDB.ASUMTAPE:
STATUS - MATCHED/MISSEDMNT/TY21ONLY status
VOLSER - Volume Serial Number
DEVNR - Device Number (a/k/a UCB Address)
DDNAME - DDNAME of first use of this volume
INITTIME - Step Initiate time of first step to use
PROGRAM - PROGRAM from first step to use volume
STEPNAME - STEPNAME from first step to use volume
ALOCSTRT - Start Time of allocation for this volume
ALOCVLTM - Duration this volume was allocated
TMNTTIME - Timestamp when Mount was issued
TAPMNTTM - Duration that it took to mount the tape
TMTDTIME - Timestamp when Mount was satisfied
TAPMTDTM - Duration when the tape was mounted
TY21TIME - Timestamp when Volume was Dismounted
BYTEREAD - Bytes Read
BYTEWRIT - Bytes Written
SIOCOUNT - SSCH/SIO Count, number of I/O operations
ERRORxxx - Tape Permanent/Temporary/Erase error counts
JOB - JOB/READTIME/JESNR, and all of the other
information from the first allocate record
and from the mount record.
To correct the errors, all you need do is to insert this
statement after your %INCLUDE SOURCLIB(BUILDPDB):
%INCLUDE SOURCLIB(ASUMTAPE);
and then use PDB.ASUMTAPE instead of PDB.TYPETMNT in your
reports and analysis programs.
If you already %INCLUDE SOURCLIB(ASUMTMNT), to create the
PDB.TAPEMNTS from PDB.TYPETMNT, you will need to add the
include of ASUMTAPE before ASUMTMNT:
%INCLUDE SOURCLIB(ASUMTAPE,ASUMTMNT);
so that the new PDB.ASUMTAPE dataset will be built and
then the revised ASUMTMNT member will create PDB.TAPEMNTS
from the new PDB.ASUMTAPE instead of from PDB.TYPETMNT.
You can correct past PDBs by running this job for each
PDB, starting with the oldest PDB first, because those
"inflight" allocations and mounts are written to the new
SPIN.SPINALOC and SPIN.SPINMOUN datasets at the end of
ASUMTAPE's execution, and are then used as input for its
next run:
// EXEC MXGSAS
//SPIN DD DSN=YOUR.SPIN.LIBRARY,DISP=OLD
//PDB DD DSN=YOUR.PDB,DISP=OLD
%INCLUDE SOURCLIB(ASUMTAPE,ASUMTMNT);
Here are the details of the errors, their cause, etc:
See additional enhancements in Change 17.077.
-MXG SMF Mount records are created for some mounts that
never happened. Jobs that are in allocation recovery
that receive WAIT and NOHOLD replies have mount records
written when message IEF433D "NOHOLD" is printed (but
there are more IEF433D messages than extra records).
Many of these extra mount records have missing JESNR or
READTIME, which are required for validation of matchup,
so those are output to READMSNG dataset and not used.
Some of these extra mount records had VOLSER='UNKNWN',
(MXGTMNT could not get the VOLSER), and originally they
were discarded until Change 17.077. UNKNWN are now used
to build PDB.ASUMTAPE, and also output in dataset UNKNWN.
just in case they are needed. The EXTRAMNT dataset
counts the additional extra mounts that were detected and
discarded in ASUMTAPE. One site had 110
UNKNWN+READMSNG+EXTRAMNT with 745 true tape mounts, while
a second had 406+507+179=1092 false mounts with 8166 true
mounts, so the impact is severe.
While just discovered, this error in MXGTMNT has probably
been there all along. We are presently unable to locate
any flags in OS/390 that can differentiate these IEF433D
mounts as false mounts; all of the bits available to us
when we sample the UCB status during the IEF433D state
look just like the bits during a real mount event, but we
are still investigating corrections to the monitor.
-MXG SMF Mount records are missed for some very fast VTS
scratch mounts, even with 1/2 second sampling. MXGTMNT
missed 219 VTS scratch mounts with 2 second sampling, but
missed only 19 with 1/2 second sampling.
Fast VTS mounts that were too fast and were missed by
MXGTMNT sampling are now captured and identified with
JOB/JESNR/etc by merging their Dismount and Allocate,
although the durations TAPMNTTM will be set to zero
and TAPMTDTM will be set to ALOCVLTM for those mounts
with STATUS='MISSEDMNT'.
-Two new code members were added and one was revised:
ASUMTAPE reads a PDB library to create the PDB.ASUMTAPE
dataset from the PDB.TYPETMNT/TYPETALO/TAPES datasets
that are build by BUILDPDB. ASUMTAPE is in JCLPDB6 so
that it will be build automatically with that JCL.
ASMFTAPE will read SMF data to create the PDB.ASUMTAPE
dataset, for sites that don't run BUILDPDB:
// EXEC MXGSAS
//SMF DD DSN=your.smf,DISP=SHR
//PDB DD DSN=the.output.library,DISP=(,CATLG)...
//SPIN DD DSN=the.inflight.spin,DISP=(,CATLG)
%INCLUDE SOURCLIB(ASMFTAPE);
ASUMTMNT creates the PDB.TAPEMNTS summary dataset that is
used in your tape mount reporting. It is now revised to
use PDB.ASUMTAPE as its input, instead of PDB.TYPETMNT,
if there are observations in PDB.ASUMTAPE. Thus you can
create the corrected PDB.ASUMTAPE dataset and use it in
place of the incorrect PDB.TYPETMNT dataset.
A print of the PDB.ASUMTAPE dataset can be produced by
the macro _TAPERPT, using this syntax:
%INCLUDE SOURCLIB(ASxxTAPE); _TAPERPT;
These WORK datasets are available for examination:
- From the TYPETMNT dataset these defective observations
were not used in the combine logic, but instead were
output to these work datasets:
READMSNG Mount missing READTIME or JESNR.
UNKNWN Mounts with VOLSER=UNKNWN, usually false
mounts, but mounts that were not opened,
like the last volume of multi-unit alloc
for scratch tape that was pre-mounted but
was not used.
- From the TYPETALO dataset these defective observations
were not used in the combine logic, but instead were
output to these work datasets:
ALLOCBAD Allocation with JESNR or JOB missing.
- From the combine of Allocate, Mount, and Dismount:
EXTRAMNT - Extra Mount or Mount with no Dismount, they
were discarded and not counted nor used.
BK2BKDIS - Back to Back Dismounts without alloc/mount.
Has not occurred in testing, but coded for.
ALLOCNOT - Allocation event that was not used; no mount
nor dismount found. Could identify jobs
that are wasting a tape drive, but most of
these are partial allocation events during
allocation recovery. A job needs two drives;
it gets the first but can't get its second.
We see the allocate event record for that
first allocate, but there was no mount nor
dismount because the job never allocated.
The job goes to the bottom of allocation
recovery, and another job is now allocated
to the drive. Typically there is a series
of back-to-back allocates for alternating
job names on the same device as jobs bounce
around.
ALLOCRET - Allocation event that was retained. Very
common to see A,M,A,A,A,A,D sequence for
a tape that was mounted once and then passed
and retained across multiple steps.
DEBUG - Zero obs unless you enable, typically for a
single device, to examine what records come
in what sequence.
SPUNMOUN - Inflight Mounts, held for tomorrow, they get
written out to SPIN.SPINMOUN.
SPUNALOC - Inflight Mounts, held for tomorrow, they get
written out to SPIN.SPINALOC.
The logic expects that a DEVNR is a unique tape drive,
i.e. no two tape devices have the same DEVNR/UCBADDRESS.
By default the variable SYSTEM is set to 'ALL ', so that
the logical merge is by DEVNR, because the IBM dismount
record can be written on a different system than the
system where the allocate and mount were written.
If you do have two tape devices with the same DEVNR
on two different systems, you can group your SMF data
for systems that share and for systems that don't and
run the ASxxTAPE program separately against each set
of SMF records, or better, you will be able to update
a new format table that MXG will use to reset the
variable SYSTEM to separate the shared and non-shared
devices. Contact Merrill to be the test site!
The implementation requires the //SPIN DD exist in your
JCL, and writes out to the //SPIN DD the SPINMOUN and
SPINALOC datasets, holding the inflight mount/allocate
records to be reintroduced in tomorrows processing.
The SPINCNT value in IMACSPIN is not used for ASUMTAPE,
because new allocations will eventually either match up
or confirm that the inflight allocate can be discarded.
I have done a lot of testing, with data from several
systems with different hardware, but this is a brand new
algorithm with lots of opportunities for validation.
I am highly confident that the PDB.ASUMTAPE matchup is
okay, and almost as confident that all other possible
non-matchups are decoded correctly into //WORK.
This program is validated for JES2 only, at present.
It does not fail with JES3, and does match the MXGTMNT
records from JES3 with the TYPE 21 records, but initial
JES3 tests suggest that because JES3 MDS allocations are
never seen by MXGTMNT, this program may not be as useful
for JES3. More JES3 testing is planned and this note
will be revised when more is known about JES3.
These sequences of Alloc/Mount/Dismount events are seen:
Normal: A M D
Multi-Volume A M D M D M D
Retained Allocation A M A A A A D
Dynamic Allocate A M D A M D
Missed Mount A D
Missed Mount Multi-Volume A D D D
Thanks to Chuck Olsen, ALLTEL, USA, whose analysis found the error.
Change 17.009 Support for deaccumulation of TYPE30_6 data. These funny
VMAC30 subtype 6 records are written for address spaces that are
Mar 4, 1999 started before SMF is up, and contain accumulated values
for most fields (but not all, and IBM doesn't document
which are accumulated and which are not) that require the
SORT and DIF() operation to produce valid interval data.
The JOB names that typically write subtype 6 records are:
ALLOCAS CONSOLE EYUX130 GRS IEFSCHAS JES2AUX JES3AUX
PCAUTH RACFDS RASP SMXC SYSBMAS TNF TRACE
VMCF
The dataset sort macro _STY30U6 for TYPE30_6 was revised
to deaccumulate the TYPE30_6 data into PDB.TYPE30_6, and
comments in VMAC30 list which variables are kept, dif()d,
which are not, and which are always missing values. Only
the PROD, ID, UREC, CPU, STOR, and PERF segments exist.
The change also corrects variables INTBTIME/INTETIME in
TYPE30_6. Because subtype 6 records contain nulls for
the SMF30IST timestamp from which GMTOFF30 is calculated,
INTB/INTE were never converted from GMT in TYPE30_6. By
revising the logic so that INTBTIME=SMF30IST only when
SMF30IST is non-missing, and by only calculating GMTOFF30
only when SMF30IST is non-missing, the GMTOFF30 from the
prior type 30 (non-subtype-6) can be used to convert GMT.
Thanks to Ron Hackett, US West, USA.
Change 17.008 Cosmetic. The &MACSHFT statement was moved to the end of
IMACSHFT the member.
Mar 4, 1999
Thanks to Chuck Hopf, MBNA, USA.
Change 17.007 TYPE50 dataset. Input should be BSIZE &PIB.4. instead of
VMAC50 BSIZE ?? 4. and the input of MXTRSIZE should be @75+
Mar 2, 1999 rather than @74+.
Thanks to Peter Durrant, National Australia Bank, AUSTRALIA.
Change 17.006 Support for DB2 type 102 subtype 199 adds QW0199xx fields
VMAC102 that report DB2 Dataset I/O statistics (avg/min/max delay
Mar 2, 1999 time in millisecs for sync/async/cached I/O plus counts).
There is an observation for every Piece or Partition of
every DB2 dataset (DBID/OBID), but only if the piece/part
averaged more than one I/O per second in the interval.
Looks interesting, and I too have used a one-I/O-per-sec
threshold to separate the wheat from the chaff.
One test record out of ten has QW0199xx fields missing
because it has only two segments (QWHSNSDA=2) when
there should be three. To be investigated with IBM.
Thanks to Dennis Pugh, BMC, USA.
Change 17.005 In the "Print first 50 obs of every NTSMF dataset" macro
VMACNTSM _RNTPRNV, the PROC PRINT DATA=_LNTDDB2 should be _LNTDB2.
Mar 1, 1999
Thanks to Mike Kynch, International Paper, USA.
Change 17.004 SYNCSORT variable INVOKSAS was not always set; the test
VMACSYNC should be IF INVOKFLG='...1....'B THEN INVOKSAS='Y'; as
Mar 1, 1999 the value of CMFSAS are x'10' or x'11'.
Thanks to Bruce Christopher, Household International, USA.
Change 17.003 Support for Candle V400 Omegamon for CICS Epilog records.
VMACEPIL Change IF OMPRDVER=451 OR OMPRDVER=450 THEN DO; to read
Feb 24, 1999 IF OMPRDVER=451 OR OMPRDVER=450 OR OMPRDVER=400 THEN DO;
and add MACRO _EPILOFF OFFEPIL=4; % in member IMACKEEP,
as there is an extra four byte length field at the start
of the V400 record.
Thanks to Malcolm Campbell, Marks and Spencer, ENGLAND
Change 17.002 The TYPEUNIC support for CA UniCenter is not for Unix,
TYPEUNIC but is for the Open VMS or VMS operating system, and test
Feb 22, 1999 data validated was from VMS 7.0. Also, the suffix is
UNIC (UNCI was typoed in change 16.391 and Newsletter).
The code is fine, I just failed to realize the data was
from VMS (but I
Change 17.001 The optional IMAC6ESS for the type 6 ESS segment was fine
IMAC6ESS under MVS, but the $VARYING fields were not converted to
Feb 22, 1999 EBCDIC when run under ASCII SAS. The INPUT and TRANSLATE
functions were added to correct.
LASTCHANGE: Version 17.
=========================member=CHANGE16================================
/* COPYRIGHT (C) 1984-1999 MERRILL CONSULTANTS DALLAS TEXAS USA */
MXG Version 16.16 is dated Feb 20, 1999, thru Change 16.394.
Newsletter THIRTY-FIVE was dated Feb 20, 1999, thru Change 16.367.
First MXG Version 16.10 was dated Feb 8, 1999, thru Change 16.364.
MXG Version 16.09 was dated Jan 20, 1999, thru Change 16.334.
MXG Version 16.08 was dated Dec 23, 1998, thru Change 16.315.
Last MXG Version 16.07 was dated Dec 5, 1998, thru Change 16.299.
First MXG Version 16.07 was dated Dec 4, 1998, thru Change 16.296.
Last MXG Version 16.06 was dated Dec 2, 1998, thru Change 16.292.
Fifth MXG Version 16.06 was dated Nov 30, 1998, thru Change 16.289.
Fourth MXG Version 16.06 was dated Nov 21, 1998, thru Change 16.285.
Third MXG Version 16.06 was dated Nov 18, 1998, thru Change 16.284.
Second MXG Version 16.06 was dated Nov 17, 1998, thru Change 16.279.
First MXG Version 16.06 was dated Nov 16, 1998, thru Change 16.278.
MXG Version 16.05 was dated Nov 1, 1998, thru Change 16.259.
Fourth MXG Version 16.04 was dated Oct 19, 1998, thru Change 16.242.
Third MXG Version 16.04 was dated Oct 9, 1998, thru Change 16.231.
Second MXG Version 16.04 was dated Oct 7, 1998, thru Change 16.224.
Newsletter THIRTY-FOUR was dated Sep 30, 1998, thru Change 16.221.
First MXG Version 16.04 was dated Sep 30, 1998, thru Change 16.221.
Beta 6 MXG Version 16.03 was dated Sep 23, 1998, thru Change 16.216.
Beta 5 MXG Version 16.03 was dated Sep 9, 1998, thru Change 16.196.
Beta 4 MXG Version 16.03 was dated Aug 11, 1998, thru Change 16.171.
Beta 3 MXG Version 16.03 was dated Jul 15, 1998, thru Change 16.147.
Beta 2 MXG Version 16.03 was dated Jul 14, 1998, thru Change 16.147.
Beta 1 MXG Version 16.03 was dated Jul 10, 1998, thru Change 16.147.
Final MXG Version 16.02 was dated Jun 8, 1998, thru Change 16.120.
First MXG Version 16.02 was dated Jun 2, 1998, thru Change 16.112.
Real MXG Version 16.01 was dated Apr 9, 1998, thru Change 16.053.
First MXG Version 16.01 was dated Apr 8, 1998, thru Change 16.052.
MXG Version 15.15 was dated Feb 23, 1998, thru Change 15.391.
Newsletter THIRTY-THREE was dated Feb 23, 1998, thru Change 15.382.
Contents of member CHANGES:
I. MXG Software Version 16.16 is now available.
II. MXG Technical Notes
III. MVS Technical Notes.
IV. DB2 Technical Notes.
V. IMS Technical Notes.
VI. SAS Technical Notes.
VII. CICS Technical Notes.
VIII. Windows NT Technical Notes.
IX. Incompatibilities and Installation of MXG 16.16.
X. Online Documentation of MXG Software.
XI. Changes Log
I. MXG Software Version 16.16 was shipped with Newsletter THIRTY-FIVE.
1. Major enhancements added in MXG 16.16 (not printed in Newsletter 35)
Support for CA UniCenter Unix Performance Monitor.
Support for TMON for CICS Version 2 PTF TH01129
Support for NETSPY Version 5.2 (COMPAT).
Support for new SNA BASE and IBM SNA objects.
ANALHSM rewritten as %MACRO, new thread use/queue measures.
XSUM70PR created to not-count ICF partition's CPU as a CPU.
Trending member for PDB.SMFINTRV / TYPE30_V dataset.
Automatically tailor BUILDPDB with the new UTILBLDP utility.
CPU increase (16.06-16.10) in BUILDPDB due to CICINTRV corrected.
Major enhancements added in MXG 16.10:
OS/390 R2.7, CICS TS 1.3 and DB2 6.1 were supported in MXG 16.09
and that support is now documented. See 16.09 enhancements.
Support for IDMS Version 14 Journal Records.
Support for Sterling's SOLVE NetMaster TCP/IP SMF.
Support for BETA93 subtypes 1,2,3,4,20,40,41,42.
Support for WINDOWS NT 5.0 (INCOMPATIBLE) with NTSMF.
Support for new NT 4.0 objects.
Major enhancements added in MXG 16.09 (revised):
Support for Year 2000 kept julian date fields input as 0cyyddd.
Support for CICS TS 1.3 (INCOMPATIBLE):
Lots of new CICSTRAN data, including Java execution
and wait times and web statistics, and new statistics data for
each of the now-eleven CICS TCBs. See Change 16.322.
Support for DB2 Version 6.1 (COMPATIBLE).
New header identifies end user's USERID and TERMINAL, more
wait information, CPU time for Stored Procs and Triggers and
User Defined Functions (with SQL time separated). Change 16.318.
Support for OS/390 Release 2.7 (COMPATIBLE)
New Type 74-6 HFS Global/Buffer/File Statistics, Type 73/79 CPMF
mode, Type 70 Dec/Shared Processors Changed, new type 74 time of
Device-Active-Only, and OS/390 Firewall Server Log Messages are
written in new Type 109 SMF record. See Change 16.329.
Major enhancements added in MXG 16.08:
Support for APAF for more than 8 CPU Engines.
Support for BETA93 Release 3.1.3 INCOMPATIBLE subtype 0, 20.
VMXGSUM will use new INHERIT if executed under SAS Version 7.
IMS 6.1 support corrected for APPC, CPIC driven transactions.
Support for DOS/VS Power V6.3.0 (COMPATIBLE).
Revisions to RMM EDGR support - numeric date variables.
Major enhancements added in MXG 16.07:
Support for APAF 4.1 and 4.3 (INCOMPATIBLE).
Support for BETA93 Release 3.1.3 INCOMPATIBLE subtype 21.
ANALRMFR RMF reports added REPORT=XCF.
Major enhancements added in MXG 16.06:
Support for NPM 2.4, unfortunately very INCOMPATIBLY changed!
PDB.CICINTRV dataset may be real bad; please get this new code!
Support for CICS TS 1.2 Journal Format, INCOMPATIBLE.
Support for PSF/MVS Release 3.1.0 (COMPATIBLE).
Support for subtypes 17-20 (FSRTYPE=) HSM FSR records.
Support for IXFP / ICEBERG fields added by APAR L170017.
Support for Boole & Babbage MainView for CICS CMRDETL file.
Correction for SMF 118 TCP/IP logic to not use length/subtype.
MXG 16.04-16.05 only. ASUM70PR summarized to SHIFT vice HOUR.
Major enhancements added in MXG 16.05:
Fix for TMS5 multi-datasets tapes (was wrong).
Fix for IMS 6.1 Log Records (TYPEIMSA) if GMT Offset was not zero.
Support for OAM (Optical Access Method) SMF type 85 record
Support for DFSORT Release 15 (no change)
Support for Interlink TCPaccess Vers 5.2 (INCOMPAT)
Support for decompression of MainView history files.
Support for RACF "installation-defined data field".
ML-18 of MXG ASMTAPES MXGTMNT monitor fixes DSAB circular queue.
Major enhancements added in MXG 16.04:
2. MXG 16.04 is a major internal redesign of MXG, the new "MACKEEP"
architecture, which makes tailoring of MXG datasets simpler and more
powerful. You can change the output DDNAME for a large dataset with
a simple "%LET Pdddddd=DDNAME;" statement, you can put all of your
MXG changes in the single member IMACKEEP (instead of having to EDIT
an IMACxxxx for each product), or you can even tailor MXG instream
using the new "%LET MACKEEP = ;" logic, and never have to EDIT any
members into your USERID.SOURCLIB!. This redesign to externalize
all attributes of every dataset began in MXG 10.10 with the "_L,_K"
macros; now that the software is finally done, I can get back to the
rewrite of MXG documentation (but note that member DOCMXG already
exists to document the new architecture, with examples!).
And most, if not all, of your existing MXG tailoring should still
work without change!
"MACKEEP" Architecture of MXG-built SAS "Datasets"
Highlights
Everything about the creation of an MXG dataset is now
fully externalized, and ALL MXG tailoring can now be done
either by
EDIT into a single member, IMACKEEP (instead of EDITing
a separate IMAC for each product)
or
all tailoring can be done "instream" using the new syntax
%LET MACKEEP= MACRO _oldname newvalue % ;
All MXG datasets now have an &Pdddddd macro variable as the
destination DDNAME/LIBREF, so you can use %LET Pdddddd=MYDD;
in IMACKEEP or with MACKEEP to change the DD to which each
MXG dataset is written or read from.
See member DOCMXG, INSTALL, and Change 16.134 for details.
Major enhancements added in MXG 16.04:
MXG "MACKEEP" Architecture - see INSTALL, DOCMXG, Change 16.134.
Support for OS/390 Release 2.6 (COMPATIBLE).
Support for IMS Log processing for IMS 6.1.
Support for NTSMF Version 2.2.
Support for ACF2 Version 6.2 type 'V' (INCOMPATIBLE).
Dostları ilə paylaş: |