本文主要介绍在x86平台上使用debootstrap来构建龙芯平台的编译和运行环境。
本文中没有非常明显的逻辑关系，由于实践的时间跨度较长，记录的可能不完整，但可以作为实践过程的一侧面印证。如有相关需求，建议参考上一文章。如要阅读本文，请自行判断文字的表达。
CLFS的尝试
本节使用CLFS做实践，具体见参考资源1。不过，CLFS不太符合需求，因此不用。
注册错误参数：
# echo ":qemu-loongarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x02\x01:\xff\xff\xff\xff\xff\xfe\xfe\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-loongarch64:" > /proc/sys/fs/binfmt_misc/register
-bash: echo: 写错误: 无效的参数
使用chroot进入clfs环境：
cd /opt/clfs-oschroot .查看内核：
bash-5.1# uname -a
Linux localhost.localdomain 4.19.0 #1 SMP Mon Oct 19 16:18:59 UTC 2020 loongarch64 GNU/Linux
将ls文件拷贝到真实龙芯机器上，执行：
$ ./ls-lfs
-bash: ./ls-lfs: 没有那个文件或目录查看连接器：
$ file ls-lfs
ls-lfs: ELF 64-bit LSB executable, LoongArch-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-loongarch64.so.1, for GNU/Linux 4.15.0, stripped添加链接器：
# ln -s /lib64/ld.so.1 /lib64/ld-linux-loongarch64.so.1执行：
# ./ls-lfs
./ls-lfs: /lib/loongarch64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./ls-lfs)
./ls-lfs: /lib/loongarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./ls-lfs)查看当前系统C库版本：
# /lib/loongarch64-linux-gnu/libc.so.6
GNU C Library (Debian GLIBC 2.28-10.kylin.35k1.0) stable release version 2.28.
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 8.3.0.
libc ABIs: UNIQUE ABSOLUTE
For bug reporting instructions, please see:
<http://www.debian.org/Bugs/>.查看LFS环境的C库：
# /lib64/libc.so.6
GNU C Library (GNU libc) development release version 2.34.9000.
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 12.0.0 20210810 (experimental).
libc ABIs: UNIQUE ABSOLUTE
For bug reporting instructions, please see:
<https://www.gnu.org/software/libc/bugs.html>.
将真实龙芯机子的ls拷贝到lfs环境：
bash-5.1# ldd lslibcap.so.2 => /usr/lib64/libcap.so.2 (0x0000004001834000)libc.so.6 => /usr/lib64/libc.so.6 (0x0000004001844000)/lib64/ld-linux-loongarch64.so.1 => /usr/lib64/ld-linux-loongarch64.so.1 (0x00000040000018d4)
bash-5.1# ldd ls_loong libselinux.so.1 => not foundlibc.so.6 => /usr/lib64/libc.so.6 (0x0000004001834000)ld.so.1 => /usr/lib64/ld.so.1 (0x00000040019c8000)/lib64/ld.so.1 => /usr/lib64/ld-linux-loongarch64.so.1 (0x00000040000018d4)
从真实机子拷贝依赖库：
bash-5.1# ldd ls_loong libselinux.so.1 => /usr/lib64/libselinux.so.1 (0x0000004001834000)libc.so.6 => /usr/lib64/libc.so.6 (0x0000004001860000)ld.so.1 => /usr/lib64/ld.so.1 (0x00000040019f4000)libpcre2-8.so.0 => /usr/lib64/libpcre2-8.so.0 (0x0000004001a24000)libdl.so.2 => /usr/lib64/libdl.so.2 (0x0000004001a88000)/lib64/ld.so.1 => /usr/lib64/ld-linux-loongarch64.so.1 (0x00000040000018d4)
结论：不同系统编译得到的软件/命令，依赖库不同。
不同版本的qemu的测试
由于qemu编译较复杂，因此笔者使用网上可找到的可执行文件进行测试。失败的记录如下。
qemu官方 失败
https://github.com/multiarch/qemu-user-static/releases 官方 v7.2.0-1 版本。
设置，拷贝，切换，但出错：
# /usr/bin/qemu-loongarch64-static -cpu ?
la464-loongarch-cpu# echo ":qemu-loongarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x02\x01:\xff\xff\xff\xff\xff\xfe\xfe\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-loongarch64-static:" > /proc/sys/fs/binfmt_misc/register# cp /usr/bin/qemu-loongarch64-static /opt/clfs-os/bin/# chroot .
/bin/bash: error while loading shared libraries: libreadline.so.8: cannot stat shared object: Error 38删除binfmt
echo -1 > /proc/sys/fs/binfmt_misc//qemu-loongarch64
loongarch官方 失败
从龙芯官方网站下载qemu，配合CFLS使用，但失败：
# /usr/bin/qemu-loongarch64-4.1-new-world -cpu ?
Loongson-3A5000-loongarch-cpu# echo ":qemu-loongarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x02\x01:\xff\xff\xff\xff\xff\xfe\xfe\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-loongarch64-4.1-new-world:" > /proc/sys/fs/binfmt_misc/register# cp /usr/bin/qemu-loongarch64-4.1-new-world /opt/clfs-os/bin/echo -1 > /proc/sys/fs/binfmt_misc/qemu-loongarch64# chroot .
/bin/bash: error while loading shared libraries: libreadline.so.8: cannot stat shared object: Error 38删除binfmt
echo -1 > /proc/sys/fs/binfmt_misc//qemu-loongarch64
永久注册 没有update-binfmts命令，失败
网上也提到一种永久注册binfmts的方式，不过测试发现失败了，可能是所用系统版本有关。
# cat >/tmp/qemu-loongarch64 <<EOF
package qemu-user-static
type magic
offset 0
magic \x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x02\x01
mask \xff\xff\xff\xff\xff\xff\xff\xfc\x00\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff
interpreter /usr/bin/qemu-loongarch64-static
EOF# update-binfmts --import /tmp/qemu-loongarch64
商业版的chroot尝试
先解压商业版本原版的iso文件，再解压filesystem.squashfs ，得到squashfs-root，此即为本节所用的系统环境。
拷贝到x86服务器上。按上述方法切换。失败告终。记录如下：
# echo -1 > /proc/sys/fs/binfmt_misc/qemu-loongarch64# echo ":qemu-loongarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x02\x01:\xff\xff\xff\xff\xff\xfe\xfe\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-loongarch64-static:" > /proc/sys/fs/binfmt_misc/register# cp /usr/bin/qemu-loongarch64-static squashfs-root/bin/# chroot squashfs-root/bin/bash: error while loading shared libraries: libtinfo.so.6: cannot stat shared object: Error 38lfs的qemu
# echo ":qemu-loongarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x02\x01:\xff\xff\xff\xff\xff\xfe\xfe\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-loongarch64:" > /proc/sys/fs/binfmt_misc/register# chroot squashfs-root
root@localhost:/# exit
个人的自问自答：
网上下载的qemu，是否是新世界的？商业版是旧世界，所以不兼容，无法跑。 网上下载的qemu，配置网上的clfs系统，可跑，但在商业版本编译的二进制，在clfs也能跑。 怎么判断clfs是哪个世界？旧版本工具链，用新的qemu跑，也会报错 cannot stat shared object: Error 38，这是否说明 qemu 有问题？
下面使用readelf命令分析各种环境的二进制文件的ELF头部数据。但除知道龙芯的elf标志flags为0x03外，限于个人能力，没有深挖下去。
龙芯CLFS的bash-5.1# readelf -h /bin/bash
ELF Header:Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 Class: ELF64Data: 2's complement, little endianVersion: 1 (current)OS/ABI: UNIX - System VABI Version: 0Type: EXEC (Executable file)Machine: LoongArchVersion: 0x1Entry point address: 0x120020948Start of program headers: 64 (bytes into file)Start of section headers: 885632 (bytes into file)Flags: 0x3Size of this header: 64 (bytes)Size of program headers: 56 (bytes)Number of program headers: 9Size of section headers: 64 (bytes)Number of section headers: 26Section header string table index: 25龙芯麒麟的：readelf -h /bin/bash
ELF 头：Magic： 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 类别: ELF64数据: 2 补码，小端序 (little endian)版本: 1 (current)OS/ABI: UNIX - System VABI 版本: 0类型: EXEC (可执行文件)系统架构: LoongArch版本: 0x1入口点地址： 0x120029540程序头起点： 64 (bytes into file)Start of section headers: 1121880 (bytes into file)标志： 0x3, LP64本头的大小： 64 (字节)程序头大小： 56 (字节)Number of program headers: 9节头大小： 64 (字节)节头数量： 27字符串表索引节头： 26龙芯麒麟编译，但在clfs的：
# readelf -h c_basic_test_loong.out
ELF Header:Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 Class: ELF64Data: 2's complement, little endianVersion: 1 (current)OS/ABI: UNIX - System VABI Version: 0Type: EXEC (Executable file)Machine: LoongArchVersion: 0x1Entry point address: 0x120000df0Start of program headers: 64 (bytes into file)Start of section headers: 20512 (bytes into file)Flags: 0x3Size of this header: 64 (bytes)Size of program headers: 56 (bytes)Number of program headers: 9Size of section headers: 64 (bytes)Number of section headers: 30Section header string table index: 29x86的：
# readelf -h /bin/bash
ELF 头：Magic： 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 类别: ELF64数据: 2 补码，小端序 (little endian)Version: 1 (current)OS/ABI: UNIX - System VABI 版本: 0类型: DYN (共享目标文件)系统架构: Advanced Micro Devices X86-64版本: 0x1入口点地址： 0x32680程序头起点： 64 (bytes into file)Start of section headers: 1165160 (bytes into file)标志： 0x0Size of this header: 64 (bytes)Size of program headers: 56 (bytes)Number of program headers: 11Size of section headers: 64 (bytes)Number of section headers: 28Section header string table index: 27
部分龙芯平台容器测试
在龙芯上成功的：
docker run -it -d --name nginx cr.loongnix.cn/library/nginx:1.24docker run -it --rm cr.loongnix.cn/library/nginx:1.24 bash
在x86上运行：
用lfs的qemu，还是exit，原因应该一样
docker run -it --rm -v /usr/bin/qemu-loongarch64:/usr/bin/qemu-loongarch64 cr.loongnix.cn/library/nginx:1.24 bash用官方qemu，提示 exec /docker-entrypoint.sh: no such file or directory
docker run -it --rm -v /usr/bin/qemu-loongarch64-static:/usr/bin/qemu-loongarch64-static cr.loongnix.cn/library/nginx:1.24 bash
loongix系统指定kylin源
在构建chroot环境时，尝试将源改为商业版本。不同厂商的源，可能不能通用，会验证公钥。即使解决公钥问题，基础的环境不同，从源安装不一定能成功。因此，最终没有采用。
vim etc/apt/sources.list.d/sources.listdeb http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1 main restricted universe multiverse
deb http://archive2.kylinos.cn/DEB/KYLIN_DEB V10-SP1 main all
deb http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1-2403-updates main restricted universe multiverse更新apt-get update
Get:1 http://archive2.kylinos.cn/DEB/KYLIN_DEB V10-SP1 InRelease [13.9 kB]
Get:2 http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1 InRelease [79.2 kB]
Get:3 http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1-2403-updates InRelease [93.8 kB] Err:1 http://archive2.kylinos.cn/DEB/KYLIN_DEB V10-SP1 InRelease The following signatures couldn't be verified because the public key is not available: NO_PUBKEY F49EC40DDCE76770
Err:2 http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1 InRelease The following signatures couldn't be verified because the public key is not available: NO_PUBKEY F49EC40DDCE76770
Err:3 http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1-2403-updates InRelease The following signatures couldn't be verified because the public key is not available: NO_PUBKEY F49EC40DDCE76770
Reading package lists... Done
W: GPG error: http://archive2.kylinos.cn/DEB/KYLIN_DEB V10-SP1 InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY F49EC40DDCE76770
E: The repository 'http://archive2.kylinos.cn/DEB/KYLIN_DEB V10-SP1 InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: GPG error: http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1 InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY F49EC40DDCE76770
E: The repository 'http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1 InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: GPG error: http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1-2403-updates InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY F49EC40DDCE76770
E: The repository 'http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1-2403-updates InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
解决（有先后）：
# apt-get install debian-archive-keyring 已安装，但依然出错# gpg --armor --export F49EC40DDCE76770 | apt-key add - # 失败
gpg: WARNING: nothing exported
gpg: no valid OpenPGP data found.# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F49EC40DDCE76770
Executing: /tmp/apt-key-gpghome.oDPCOIrEWv/gpg.1.sh --keyserver keyserver.ubuntu.com --recv-keys F49EC40DDCE76770
gpg: key F49EC40DDCE76770: public key "Kylin Archive Automatic Signing Key (For Kylin Arm64 Repo.) <kylin_devel@kylinos.cn>" imported
gpg: Total number processed: 1
gpg: imported: 1更新成功：
root@localhost:/# apt-get update
Hit:1 http://archive2.kylinos.cn/DEB/KYLIN_DEB V10-SP1 InRelease
Hit:2 http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1-2403-updates InRelease
Get:3 http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1 InRelease [79.2 kB]
Get:4 http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1/main loongarch64 Packages [3270 kB]
Get:5 http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1/restricted loongarch64 Packages [1247 B]
Get:6 http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1/universe loongarch64 Packages [16.7 MB]
Get:7 http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1/multiverse loongarch64 Packages [166 kB]
Fetched 20.3 MB in 43s (470 kB/s)
Reading package lists... Done
sudo apt-key adv --fetch-keys 'http://pkg.iloongarch.cn/loongnix20/key/public.key'
loongix系统指定debian源 未做
除上述商业版的源外，也可指定其它发行版本的，不过限于时间，没有实践。
# cat /srv/chroots/sid-loong64-sbuild/etc/apt/sources.list
# binary default
deb http://ftp.ports.debian.org/debian-ports unstable main
deb http://ftp.ports.debian.org/debian-ports unreleased main# source
deb-src http://ftp.cn.debian.org/debian/ sid main
地址：
http://ftp.ports.debian.org/debian-ports/
使用kylin发行版本不成功的尝试
思路
分析第一阶段的下载命令，核心是指定安装源、指定发行版本。考虑到运行程序的系统是kylin，而不是loongnix，所以尝试改成kylin版本。
为防止意外，本次实验解析了kylin的光盘镜像备用。
第一阶段：下载
参考loongnix的下载命令，修改成下载kylin源的：
# debootstrap --no-check-gpg --variant=minbase --components=main,non-free,contrib --arch=loongarch64 --foreign 10.1 kylin_rootfs https://archive.kylinos.cn/kylin/KYLIN-ALL
发行版本变成10.1（地址：https://archive.kylinos.cn/kylin/KYLIN-ALL/dists/10.1/），软件源变成https://archive.kylinos.cn/kylin/KYLIN-ALL。
输出：
I: Retrieving InRelease
...
I: Extracting util-linux...
I: Extracting zlib1g...
第二阶段：安装并切换
chroot iso debootstrap/debootstrap --second-stage
此阶段出错较多，需根据错误信息解决问题，如缺少库的，则从光盘镜像目录里拷贝之。
以下是缺少库的输出信息：
/bin/sh: error while loading shared libraries: libkysec.so.0: cannot open shared object file: No such file or directory
/bin/sh: error while loading shared libraries: libkysec_log.so.0: cannot open shared object file: No such file or directory
/usr/bin/mkdir: error while loading shared libraries: libpcre2-8.so.0: cannot open shared object file: No such file or directory/bin/dpkg-deb: error while loading shared libraries: libzstd.so.1: cannot open shared object file: No such file or directory
拷贝示例：
cp -a squashfs-root/usr/lib/loongarch64-linux-gnu/libzstd.so.1 kylin_rootfs /usr/lib/loongarch64-linux-gnu/
拷贝之后，没有缺少库的提示，但dpkg出错了：
dpkg: error: parsing file '/var/lib/dpkg/status' near line 2 package 'dpkg':
尝试删除iso/var/lib/dpkg/status文件，再试，依然出错：
W: Failure while configuring required packages.
W: See //debootstrap/debootstrap.log for details (possibly the package passwd is at fault)
重复多次，仍无法解决，逐放弃。
小结
本文列出一些失败的尝试，由于是失败的，如果要有总结，那就是积累了一定的经验，排除掉了一些不合用的备选项。
参考资源
- 龙芯clfs：https://github.com/sunhaiyong1978/CLFS-for-LoongArch/blob/main/Qemu_For_LoongArch64-Simple.md 。
