Compile drbd9 on CentOS7

背景:最近CentOS7系统升级到最新内核后,使用从yum源中安装的drbd9导致内核崩溃

[root@lms-1 ~]# uname -a
Linux lms-1 3.10.0-862.2.3.el7.x86_64 #1 SMP Wed May 9 18:05:47 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@lms-1 ~]# yum list installed | grep drbd
drbd90-utils.x86_64                        9.1.0-1.el7.elrepo          @elrepo
kmod-drbd90.x86_64                         9.0.9-1.el7_4.elrepo        @elrepo

崩溃信息

[  124.792108] drbd r0: Starting worker thread (from drbdsetup [3092])
[  124.795286] ------------[ cut here ]------------
[  124.795873] kernel BUG at lib/nlattr.c:66!
[  124.796362] invalid opcode: 0000 [#1] SMP
[  124.796880] Modules linked in: drbd(OE) ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_filter veth xfs bridge stp llc dm_thin_pooldm_persistent_data dm_bio_prison dm_bufio libcrc32c loop iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev parport_pc sg aesni_intel lrw gf128mul glue_helper ablk_helper cryptd parport joydev pcspkr virtio_balloon i2c_piix4 ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_scsi virtio_net cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw i2c_core virtio_pci crct10dif_pclmul crct10dif_common virtio_ring crc32c_intel floppy virtio dm_mirror dm_region_hash dm_log dm_mod
[  124.805782] CPU: 1 PID: 3097 Comm: drbdsetup Tainted: G           OE  ------------   3.10.0-862.2.3.el7.x86_64 #1
[  124.806978] Hardware name: Red Hat KVM, BIOS seabios-1.7.5-11.el7 04/01/2014
[  124.807796] task: ffff89f72df4eeb0 ti: ffff89f4b0278000 task.ti: ffff89f4b0278000
[  124.808684] RIP: 0010:[<ffffffff86176a80>]  [<ffffffff86176a80>] validate_nla+0x230/0x240
[  124.809661] RSP: 0018:ffff89f4b027b928  EFLAGS: 00010202
[  124.810284] RAX: 00000000000001f0 RBX: ffff89f778235c34 RCX: 0000000000003720
[  124.811122] RDX: ffffffffc0466f20 RSI: 0000000000000024 RDI: 000000000000000a
[  124.811952] RBP: ffff89f4b027b948 R08: ffffffffc0466f20 R09: ffffffffc04674a0
[  124.815392] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  124.816063] CR2: 00007f6ec5067f40 CR3: 00000000702be000 CR4: 00000000003606e0
[  124.816899] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  124.817726] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  124.818558] Call Trace:
[  124.818859]  [<ffffffff86176bc6>] nla_parse+0xb6/0x120
[  124.819470]  [<ffffffffc044ff53>] drbd_nla_parse_nested+0x43/0x50 [drbd]
[  124.820259]  [<ffffffffc0433c47>] __net_conf_from_attrs.isra.60+0x37/0x940 [drbd]
[  124.821167]  [<ffffffffc044131a>] adm_new_connection+0x9a/0x8e0 [drbd]
[  124.821944]  [<ffffffff86176bc6>] ? nla_parse+0xb6/0x120
[  124.822569]  [<ffffffff86176f24>] ? __nla_reserve_nohdr+0x24/0x30
[  124.823286]  [<ffffffffc0441c10>] drbd_adm_new_peer+0xb0/0xc0 [drbd]
[  124.824035]  [<ffffffff86426c38>] genl_family_rcv_msg+0x208/0x430
[  124.824745]  [<ffffffff86151324>] ? __radix_tree_lookup+0x84/0xf0
[  124.825459]  [<ffffffff86426ebb>] genl_rcv_msg+0x5b/0xc0
[  124.826085]  [<ffffffff864231a0>] ? __netlink_lookup+0xc0/0x110
[  124.826784]  [<ffffffff86426e60>] ? genl_family_rcv_msg+0x430/0x430
[  124.827518]  [<ffffffff86424ecb>] netlink_rcv_skb+0xab/0xc0
[  124.828172]  [<ffffffff86425408>] genl_rcv+0x28/0x40
[  124.828755]  [<ffffffff86424850>] netlink_unicast+0x170/0x210
[  124.829431]  [<ffffffff86424bf8>] netlink_sendmsg+0x308/0x420
[  124.830104]  [<ffffffff863cb84d>] sock_aio_write+0x15d/0x180
[  124.830771]  [<ffffffff85fc527c>] ? handle_pte_fault+0x2dc/0xc30
[  124.831471]  [<ffffffff8601a193>] do_sync_write+0x93/0xe0
[  124.832105]  [<ffffffff8601ad75>] vfs_write+0x1c5/0x1f0
[  124.832717]  [<ffffffff8601ba9f>] SyS_write+0x7f/0xf0
[  124.833314]  [<ffffffff8651f795>] system_call_fastpath+0x1c/0x21
[  124.834014] Code: 5e 0f 4f c2 5d c3 0f 1f 44 00 00 31 d2 66 85 c9 74 de e9 e4 fe ff ff b8 ea ff ff ff e9 3c fe ff ff b8 de ff ff ff e9 32 fe ff ff <0f> 0b 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 55 83 fe 03 48
[  124.837228] RIP  [<ffffffff86176a80>] validate_nla+0x230/0x240
[  124.837925]  RSP <ffff89f4b027b928>
[  124.838356] ---[ end trace 43a47828d486244d ]---
[  124.838903] Kernel panic - not syncing: Fatal exception
[  124.839568] Kernel Offset: 0x4e00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)

随后决定自己编译安装,下面是制作rpm包的过程

编译环境

OS:CentOS7
Kernel :3.10.0-862.2.3

下载软件包

https://www.linbit.com/en/drbd-community/drbd-download/
drbd-9.0.14-1.tar
drbd-utils-9.4.0.tar

升级内核

yum install kernel kernel-devel kernel-headers

安装编译需要的工具

yum install gcc make automake gcc-c++ flex

编译drbd内核模块

tar xzvf drbd-9.0.14-1.tar
cd /drbd-9.0.14-1
make
...
    Module build was successful.
=======================================================================
  With DRBD module version 8.4.5, we split out the management tools
  into their own repository at https://github.com/LINBIT/drbd-utils
  (tarball at http://links.linbit.com/drbd-download)

  That started out as "drbd-utils version 8.9.0",
  has a different release cycle,
  and provides compatible drbdadm, drbdsetup and drbdmeta tools
  for DRBD module versions 8.3, 8.4 and 9.

  Again: to manage DRBD 9 kernel modules and above,
  you want drbd-utils >= 9.3 from above url.
=======================================================================

制作rpm包

make kmp-rpm

编译drbd-utils

tar xzvf drbd-utils-9.4.0.tar
cd drbd-utils-9.4.0
直接生成drbd.spec文件,制作rpm包,因为源码bug的关系,需要手动建立/root/rpmbuild/SOURCES目录
mkdir /root/rpmbuild/SOURCES
生成Makefile文件
./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc --without-83support --without-84support
生成drbd.spec文件
./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc --without-83support --without-84support --enable-spec
make rpm
...
You have now:
/root/rpmbuild/RPMS/x86_64/drbd-9.4.0-1.el7.centos.x86_64.rpm
/root/rpmbuild/RPMS/x86_64/drbd-utils-9.4.0-1.el7.centos.x86_64.rpm
/root/rpmbuild/RPMS/x86_64/drbd-xen-9.4.0-1.el7.centos.x86_64.rpm
/root/rpmbuild/RPMS/x86_64/drbd-udev-9.4.0-1.el7.centos.x86_64.rpm
/root/rpmbuild/RPMS/x86_64/drbd-pacemaker-9.4.0-1.el7.centos.x86_64.rpm
/root/rpmbuild/RPMS/x86_64/drbd-heartbeat-9.4.0-1.el7.centos.x86_64.rpm
/root/rpmbuild/RPMS/x86_64/drbd-bash-completion-9.4.0-1.el7.centos.x86_64.rpm
/root/rpmbuild/RPMS/x86_64/drbd-man-ja-9.4.0-1.el7.centos.x86_64.rpm
/root/rpmbuild/RPMS/x86_64/drbd-debuginfo-9.4.0-1.el7.centos.x86_64.rpm

这样编译出来的包安装会可能会出现以下问题:

[root@lms-0 ~]#  rpm -ivh drbd-utils-9.4.0-1.el7.centos.x86_64.rpm
Preparing...                          ################################# [100%]
        file /usr/sbin/drbdadm conflicts between attempted installs of drbd-utils-9.4.0-1.el7.centos.x86_64 and drbd-utils-9.4.0-1.el7.centos.x86_64
        file /usr/sbin/drbdmeta conflicts between attempted installs of drbd-utils-9.4.0-1.el7.centos.x86_64 and drbd-utils-9.4.0-1.el7.centos.x86_64
        file /usr/sbin/drbdsetup conflicts between attempted installs of drbd-utils-9.4.0-1.el7.centos.x86_64 and drbd-utils-9.4.0-1.el7.centos.x86_64

解决办法:修改drbd.spec.in文件

  # conditionals may not contain "-" nor "_", hence "bashcompletion"
  %bcond_without bashcompletion
  %bcond_without sbinsymlinks
+ %if %{rhel} == 7
+ %undefine with_sbinsymlinks
+ %endif
  # --with xen is ignored on any non-x86 architecture
  %bcond_without xen
  %bcond_without 83support

重新打包,安装成功

参考

http://tldp.org/HOWTO/RPM-HOWTO/build.html
http://fedoraproject.org/wiki/Packaging:RPMMacros
https://docs.linbit.com/doc/users-guide-84/ch-build-install-from-source/
http://lists.linbit.com/pipermail/drbd-user/2015-November/021922.html