User Rating: 5 / 5

Star ActiveStar ActiveStar ActiveStar ActiveStar Active
 

This quick article will test ASM Filter Driver i/o filtering out non-oracle operation. So we will try to overwrite ASM disk with some common linux commands.

Before starting, you need ...
... Grid Infrastructure 12c already installed with ASMFD (if not, check this)
... a root access

Warning ...
Be aware that some commands in this article may DESTROY your data. So make sure you know what you are doing, do it on your own responsibility.

About Oracle ASM Filter Driver

Oracle ASM Filter Driver (Oracle ASMFD) is a kernel module that resides in the I/O path of the Oracle ASM disks. Oracle ASM uses the filter driver to validate write I/O requests to Oracle ASM disks.

Oracle ASMFD simplifies the configuration and management of disk devices by eliminating the need to rebind disk devices used with Oracle ASM each time the system is restarted.

Oracle ASM Filter Driver rejects any I/O requests that are invalid. This action eliminates accidental overwrites of Oracle ASM disks that would cause corruption in the disks and files within the disk group. For example, the Oracle ASM Filter Driver filters out all non-Oracle I/Os which could cause accidental overwrites.

docs.oracle.com

Check if ASM Filtering is enabled

We need to make sure that ASMFD is loaded and filtering is enabled on the disk we gonna try to corrupt (/dev/sda).

# Check if ASMFD is loaded
[grid]$ asmcmd afd_state
    ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'oralab01.uxora.com'

# List ASM Disk with filtering enabled
[grid]$ asmcmd afd_lsdsk
    ---------------------------------------------------------
    Label                     Filtering   Path
    =========================================================
    DISK01                      ENABLED   /dev/sda
    DISK02                      ENABLED   /dev/sdb
    DISK03                      ENABLED   /dev/sdc
    DISK04                      ENABLED   /dev/sdd

Disk manipulation

As root user, we are going to do disk manipulation on /dev/sda which is an asm disk with filtering enabled.
First, we going to read the first bytes of /dev/sda :

# Get header
[root]$ od -c -N 128 /dev/sda
    0000000 001 202 001 001  \0  \0  \0  \0  \0  \0  \0 200 247   @ 203 220
    0000020  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
    0000040   O   R   C   L   D   I   S   K   D   I   S   K   0   1  \0  \0
    0000060  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
    0000100  \0 001      \f  \0  \0 001 003   D   I   S   K   0   1  \0  \0
    0000120  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
    0000140  \0  \0  \0  \0  \0  \0  \0  \0   D   A   T   A  \0  \0  \0  \0
    0000160  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
    0000200

fdisk

# Try partition with fdisk
[root]$ fdisk /dev/sda
    Welcome to fdisk (util-linux 2.23.2).

    Changes will remain in memory only, until you decide to write them.
    Be careful before using the write command.

    Device does not contain a recognized partition table
    Building a new DOS disklabel with disk identifier 0xd2fcbd9d.

    Command (m for help): n
    Partition type:
       p   primary (0 primary, 0 extended, 4 free)
       e   extended
    Select (default p):
    Using default response p
    Partition number (1-4, default 1):
    First sector (2048-50331647, default 2048):
    Using default value 2048
    Last sector, +sectors or +size{K,M,G} (2048-50331647, default 50331647):
    Using default value 50331647
    Partition 1 of type Linux and of size 24 GiB is set

    Command (m for help): w
    The partition table has been altered!

    Calling ioctl() to re-read partition table.

    Error closing file

# Check /var/log/message for error messages
[root]$ tail -f /var/log/message
    ...
    Aug 19 22:05:28 oralab01 kernel: Buffer I/O error on dev sda, logical block 0, lost async page write
    Aug 19 22:05:28 oralab01 kernel: F 4297671.556/170819200528 fdisk[13113] oracleafd:18:0894:Write IO to ASM managed device: [8] [0]
    ...

mkfs

# Try to create filesystem with mkfs
[root]$ mkfs.xfs -f /dev/sda
    meta-data=/dev/sda               isize=256    agcount=4, agsize=1572864 blks
             =                       sectsz=512   attr=2, projid32bit=1
             =                       crc=0        finobt=0, sparse=0
    data     =                       bsize=4096   blocks=6291456, imaxpct=25
             =                       sunit=0      swidth=0 blks
    naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
    log      =internal log           bsize=4096   blocks=3072, version=2
             =                       sectsz=512   sunit=0 blks, lazy-count=1
    realtime =none                   extsz=4096   blocks=0, rtextents=0
    mkfs.xfs: pwrite64 failed: Input/output error

# Check /var/log/message for error messages
[root]$ tail -f /var/log/message
    ...
    Aug 19 22:24:18 oralab01 kernel: F 4306001.982/170819222418 mkfs.xfs[11228] oracleafd:18:0894:Write IO to ASM managed device: [8] [0]
    ...

dd

# Try to erase header with dd
[root]$ dd if=/dev/zero of=/dev/sda bs=4096 count=1000
    1000+0 records in
    1000+0 records out
    4096000 bytes (4.1 MB) copied, 0.00493748 s, 830 MB/s

# Check /var/log/message for error messages
[root]$ tail -f /var/log/message
    ...
    Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 0, lost async page write
    Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 1, lost async page write
    Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 2, lost async page write
    Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 3, lost async page write
    Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 4, lost async page write
    Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 5, lost async page write
    Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 6, lost async page write
    Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 7, lost async page write
    Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 8, lost async page write
    Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 9, lost async page write
    ...

# Try to erase header with dd direct io
[root]$ dd if=/dev/zero of=/dev/sda bs=4096 count=1000 oflag=direct
    dd: error writing ‘/dev/sda’: Input/output error
    1+0 records in
    0+0 records out
    0 bytes (0 B) copied, 0.000186557 s, 0.0 kB/s

# Check /var/log/message for error messages
[root]$ tail -f /var/log/message
    ...
    Aug 19 22:16:41 oralab01 kernel: F 4298344.799/170819201641 dd[20904] oracleafd:18:0894:Write IO to ASM managed device: [8] [0]
    ...

io redirection

# Try to erase header with io redirection
[root]$ echo HELLOWORLD > /dev/sda

# Check /var/log/message for error messages
[root]$ tail -f /var/log/message
    ...
    Aug 19 21:55:17 oralab01 kernel: Buffer I/O error on dev sda, logical block 0, lost async page write
    ...

Check ASM Disk

After all theses disks manipulations, let's check /dev/sda disk.

Re-Read header with od

Header still is the same before and after disk manipulations.

# Get header
[root]$ od -c -N 128 /dev/sda
    0000000 001 202 001 001  \0  \0  \0  \0  \0  \0  \0 200 247   @ 203 220
    0000020  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
    0000040   O   R   C   L   D   I   S   K   D   I   S   K   0   1  \0  \0
    0000060  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
    0000100  \0 001      \f  \0  \0 001 003   D   I   S   K   0   1  \0  \0
    0000120  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
    0000140  \0  \0  \0  \0  \0  \0  \0  \0   D   A   T   A  \0  \0  \0  \0
    0000160  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
    0000200

Read header with kfed

Kfed can still read the asm disk header.

# Header with kfed
[grid]$ kfed read /dev/sda
    kfbh.endian:                          1 ; 0x000: 0x01
    kfbh.hard:                          130 ; 0x001: 0x82
    kfbh.type:                            1 ; 0x002: KFBTYP_DISKHEAD
    kfbh.datfmt:                          1 ; 0x003: 0x01
    kfbh.block.blk:                       0 ; 0x004: blk=0
    kfbh.block.obj:              2147483648 ; 0x008: disk=0
    kfbh.check:                  2424520871 ; 0x00c: 0x908340a7
    kfbh.fcn.base:                        0 ; 0x010: 0x00000000
    kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000
    kfbh.spare1:                          0 ; 0x018: 0x00000000
    kfbh.spare2:                          0 ; 0x01c: 0x00000000
    kfdhdb.driver.provstr:   ORCLDISKDISK01 ; 0x000: length=14
    kfdhdb.driver.reserved[0]:   1263749444 ; 0x008: 0x4b534944
    kfdhdb.driver.reserved[1]:        12592 ; 0x00c: 0x00003130
    kfdhdb.driver.reserved[2]:            0 ; 0x010: 0x00000000
    kfdhdb.driver.reserved[3]:            0 ; 0x014: 0x00000000
    kfdhdb.driver.reserved[4]:            0 ; 0x018: 0x00000000
    kfdhdb.driver.reserved[5]:            0 ; 0x01c: 0x00000000
    kfdhdb.compat:                203424000 ; 0x020: 0x0c200100
    kfdhdb.dsknum:                        0 ; 0x024: 0x0000
    kfdhdb.grptyp:                        1 ; 0x026: KFDGTP_EXTERNAL
    kfdhdb.hdrsts:                        3 ; 0x027: KFDHDR_MEMBER
    kfdhdb.dskname:                  DISK01 ; 0x028: length=6
    kfdhdb.grpname:                    DATA ; 0x048: length=4
    kfdhdb.fgname:                   DISK01 ; 0x068: length=6
    kfdhdb.siteguid[0]:                   0 ; 0x088: 0x00
    kfdhdb.siteguid[1]:                   0 ; 0x089: 0x00
    kfdhdb.siteguid[2]:                   0 ; 0x08a: 0x00
    kfdhdb.siteguid[3]:                   0 ; 0x08b: 0x00
    kfdhdb.siteguid[4]:                   0 ; 0x08c: 0x00
    kfdhdb.siteguid[5]:                   0 ; 0x08d: 0x00
    kfdhdb.siteguid[6]:                   0 ; 0x08e: 0x00
    kfdhdb.siteguid[7]:                   0 ; 0x08f: 0x00
    kfdhdb.siteguid[8]:                   0 ; 0x090: 0x00
    kfdhdb.siteguid[9]:                   0 ; 0x091: 0x00
    kfdhdb.siteguid[10]:                  0 ; 0x092: 0x00
    kfdhdb.siteguid[11]:                  0 ; 0x093: 0x00
    kfdhdb.siteguid[12]:                  0 ; 0x094: 0x00
    kfdhdb.siteguid[13]:                  0 ; 0x095: 0x00
    kfdhdb.siteguid[14]:                  0 ; 0x096: 0x00
    kfdhdb.siteguid[15]:                  0 ; 0x097: 0x00
    kfdhdb.ub1spare[0]:                   0 ; 0x098: 0x00
    kfdhdb.ub1spare[1]:                   0 ; 0x099: 0x00
    kfdhdb.ub1spare[2]:                   0 ; 0x09a: 0x00
    kfdhdb.ub1spare[3]:                   0 ; 0x09b: 0x00
    kfdhdb.ub1spare[4]:                   0 ; 0x09c: 0x00
    kfdhdb.ub1spare[5]:                   0 ; 0x09d: 0x00
    kfdhdb.ub1spare[6]:                   0 ; 0x09e: 0x00
    kfdhdb.ub1spare[7]:                   0 ; 0x09f: 0x00
    kfdhdb.ub1spare[8]:                   0 ; 0x0a0: 0x00
    kfdhdb.ub1spare[9]:                   0 ; 0x0a1: 0x00
    kfdhdb.ub1spare[10]:                  0 ; 0x0a2: 0x00
    kfdhdb.ub1spare[11]:                  0 ; 0x0a3: 0x00
    kfdhdb.ub1spare[12]:                  0 ; 0x0a4: 0x00
    kfdhdb.ub1spare[13]:                  0 ; 0x0a5: 0x00
    kfdhdb.ub1spare[14]:                  0 ; 0x0a6: 0x00
    kfdhdb.ub1spare[15]:                  0 ; 0x0a7: 0x00
    kfdhdb.crestmp.hi:             33055335 ; 0x0a8: HOUR=0x7 DAYS=0x13 MNTH=0x8 YEAR=0x7e1
    kfdhdb.crestmp.lo:            888763392 ; 0x0ac: USEC=0x0 MSEC=0x25d SECS=0xf MINS=0xd
    kfdhdb.mntstmp.hi:             33055349 ; 0x0b0: HOUR=0x15 DAYS=0x13 MNTH=0x8 YEAR=0x7e1
    kfdhdb.mntstmp.lo:           1119166464 ; 0x0b4: USEC=0x0 MSEC=0x148 SECS=0x2b MINS=0x10
    kfdhdb.secsize:                     512 ; 0x0b8: 0x0200
    kfdhdb.blksize:                    4096 ; 0x0ba: 0x1000
    kfdhdb.ausize:                  4194304 ; 0x0bc: 0x00400000
    kfdhdb.mfact:                    454272 ; 0x0c0: 0x0006ee80
    kfdhdb.dsksize:                    6144 ; 0x0c4: 0x00001800
    kfdhdb.pmcnt:                         3 ; 0x0c8: 0x00000003
    kfdhdb.fstlocn:                       1 ; 0x0cc: 0x00000001
    kfdhdb.altlocn:                       2 ; 0x0d0: 0x00000002
    kfdhdb.f1b1locn:                     10 ; 0x0d4: 0x0000000a
    kfdhdb.redomirrors[0]:                0 ; 0x0d8: 0x0000
    kfdhdb.redomirrors[1]:                0 ; 0x0da: 0x0000
    kfdhdb.redomirrors[2]:                0 ; 0x0dc: 0x0000
    kfdhdb.redomirrors[3]:                0 ; 0x0de: 0x0000
    kfdhdb.dbcompat:              168820736 ; 0x0e0: 0x0a100000
    kfdhdb.grpstmp.hi:             33055335 ; 0x0e4: HOUR=0x7 DAYS=0x13 MNTH=0x8 YEAR=0x7e1
    kfdhdb.grpstmp.lo:            888013824 ; 0x0e8: USEC=0x0 MSEC=0x381 SECS=0xe MINS=0xd
    kfdhdb.vfstart:                      24 ; 0x0ec: 0x00000018
    kfdhdb.vfend:                        32 ; 0x0f0: 0x00000020
    kfdhdb.spfile:                        0 ; 0x0f4: 0x00000000
    kfdhdb.spfflg:                        0 ; 0x0f8: 0x00000000
    kfdhdb.flags:                         1 ; 0x0fc: 0x00000001
    kfdhdb.f1b1fcn.base:                  0 ; 0x100: 0x00000000
    kfdhdb.f1b1fcn.wrap:                  0 ; 0x104: 0x00000000
    kfdhdb.ip[0]:                       192 ; 0x108: 0xc0
    kfdhdb.ip[1]:                       168 ; 0x109: 0xa8
    kfdhdb.ip[2]:                         0 ; 0x10a: 0x00
    kfdhdb.ip[3]:                        31 ; 0x10b: 0x1f
    kfdhdb.modstmp:              1503170203 ; 0x10c: 0x59988e9b
    kfdhdb.checklbl:                      0 ; 0x110: 0x00
    kfdhdb.verlbl:                        0 ; 0x111: 0x00
    kfdhdb.ub2spare:                      0 ; 0x112: 0x0000
    kfdhdb.sitelbl:                         ; 0x114: length=0
    kfdhdb.fglbl:                           ; 0x124: length=0
    kfdhdb.vsnnum:                203424000 ; 0x144: 0x0c200100
    kfdhdb.patchvsn:                      0 ; 0x148: 0x0000
    kfdhdb.operation:                     0 ; 0x14a: 0x0000
    kfdhdb.xtnd[0]:                       0 ; 0x14c: 0x0000
    kfdhdb.xtnd[1]:                       0 ; 0x14e: 0x0000
    kfdhdb.xtnd[2]:                       0 ; 0x150: 0x0000
    kfdhdb.xtnd[3]:                       0 ; 0x152: 0x0000
    kfdhdb.xtnd[4]:                       0 ; 0x154: 0x0000
    kfdhdb.xtnd[5]:                       0 ; 0x156: 0x0000
    kfdhdb.ub4spare[0]:                   0 ; 0x158: 0x00000000
    kfdhdb.ub4spare[1]:                   0 ; 0x15c: 0x00000000
    kfdhdb.ub4spare[2]:                   0 ; 0x160: 0x00000000
    kfdhdb.ub4spare[3]:                   0 ; 0x164: 0x00000000
    kfdhdb.ub4spare[4]:                   0 ; 0x168: 0x00000000
    kfdhdb.ub4spare[5]:                   0 ; 0x16c: 0x00000000
    kfdhdb.ub4spare[6]:                   0 ; 0x170: 0x00000000
    kfdhdb.ub4spare[7]:                   0 ; 0x174: 0x00000000
    kfdhdb.ub4spare[8]:                   0 ; 0x178: 0x00000000
    kfdhdb.ub4spare[9]:                   0 ; 0x17c: 0x00000000
    kfdhdb.ub4spare[10]:                  0 ; 0x180: 0x00000000
    kfdhdb.ub4spare[11]:                  0 ; 0x184: 0x00000000
    kfdhdb.ub4spare[12]:                  0 ; 0x188: 0x00000000
    kfdhdb.ub4spare[13]:                  0 ; 0x18c: 0x00000000
    kfdhdb.ub4spare[14]:                  0 ; 0x190: 0x00000000
    kfdhdb.ub4spare[15]:                  0 ; 0x194: 0x00000000
    kfdhdb.ub4spare[16]:                  0 ; 0x198: 0x00000000
    kfdhdb.ub4spare[17]:                  0 ; 0x19c: 0x00000000
    kfdhdb.ub4spare[18]:                  0 ; 0x1a0: 0x00000000
    kfdhdb.ub4spare[19]:                  0 ; 0x1a4: 0x00000000
    kfdhdb.ub4spare[20]:                  0 ; 0x1a8: 0x00000000
    kfdhdb.ub4spare[21]:                  0 ; 0x1ac: 0x00000000
    kfdhdb.ub4spare[22]:                  0 ; 0x1b0: 0x00000000
    kfdhdb.ub4spare[23]:                  0 ; 0x1b4: 0x00000000
    kfdhdb.ub4spare[24]:                  0 ; 0x1b8: 0x00000000
    kfdhdb.ub4spare[25]:                  0 ; 0x1bc: 0x00000000
    kfdhdb.ub4spare[26]:                  0 ; 0x1c0: 0x00000000
    kfdhdb.ub4spare[27]:                  0 ; 0x1c4: 0x00000000
    kfdhdb.ub4spare[28]:                  0 ; 0x1c8: 0x00000000
    kfdhdb.ub4spare[29]:                  0 ; 0x1cc: 0x00000000
    kfdhdb.ub4spare[30]:                  0 ; 0x1d0: 0x00000000
    kfdhdb.acdb.aba.seq:                  0 ; 0x1d4: 0x00000000
    kfdhdb.acdb.aba.blk:                  0 ; 0x1d8: 0x00000000
    kfdhdb.acdb.ents:                     0 ; 0x1dc: 0x0000
    kfdhdb.acdb.ub2spare:                 0 ; 0x1de: 0x0000

Check disk with ASM

# Check datagroup with asmcmd
[grid]$ asmcmd chkdg data
    Diskgroup altered.

[grid]$ tail -20 /u01/app/grid/diag/asm/+asm/+ASM1/trace/alert_+ASM1.log
    ...
    SQL> /* ASMCMD */ALTER DISKGROUP data CHECK  NOREPAIR
    NOTE: starting check of diskgroup DATA
    GMON querying group 1 at 24 for pid 38, osid 9849
    GMON checking disk 0 for group 1 at 25 for pid 38, osid 9849
    GMON querying group 1 at 26 for pid 38, osid 9849
    GMON checking disk 1 for group 1 at 27 for pid 38, osid 9849
    GMON querying group 1 at 28 for pid 38, osid 9849
    GMON checking disk 2 for group 1 at 29 for pid 38, osid 9849
    GMON querying group 1 at 30 for pid 38, osid 9849
    GMON checking disk 3 for group 1 at 31 for pid 38, osid 9849
    SUCCESS: check of diskgroup DATA found no errors
    SUCCESS: /* ASMCMD */ALTER DISKGROUP data CHECK  NOREPAIR
    ...

Conclusion

It does what it says and It works pretty well so far!
This will protect from some wrong disk administration mistakes and avoid the dba a full db backup recovery.
So I think this is a great feature which may save a dba some stressful time.

 

Hope This Helps,
Michel.

Reference
Administering Oracle ASM Filter Driver (docs.oracle.com)
Oracle’s ASM Filter Driver Revisited (flashdba.com)

Enjoyed this article? Please like it or share it.

Add comment

Please connect with one of social login below (or fill up name and email)

     


Security code
Refresh