[solved] Self compiled python is missing basic features like readline and _posixsubprocess

相关文章推荐

帅呆的炒粉 · pytorch ...· 2 周前 ·

急躁的自行车 · 深入解读：Scikit-learn、Pyth ...· 2 周前 ·

追风的花生 · 【学习交流】python简易爬虫入门- ...· 2 周前 ·

不拘小节的牛腩 · 欢迎访问PinPong ...· 2 周前 ·

没人理的大熊猫 · 列表 List - Ant Design· 10 月前 ·

欢快的夕阳 · S3 Range Header · ...· 10 月前 ·

有情有义的大白菜 · WPF 将数据绑定到 DataGrid 的 ...· 1 年前 ·

大鼻子的大熊猫 · 如何知道微信小程序内置浏览器的 js ...· 1 年前 ·

微醺的硬币 · Java解析多帧DICOM - ...· 1 年前 ·

I would like to install self compiled python versions in my opensuse home directory.
Unfortunately many basic modules are not installed.

To achieve this, I did the following (trying to mimic instructions for other distros):

zypper in tk-devel sqlite3-devel gdbm-devel readline6-devel openssl-devel
ver=3.11.1
wget -O- https://www.python.org/ftp/python/${ver}/Python-${ver}.tar.xz | tar xJf -
cd Python-${ver}
./configure --prefix=$PWD/run
make -j8
make install
Now this most basic check works:
~/Python-3.11.1/run/bin/python3 --version
Python 3.11.1
Mostly anything else fails (interactive mode misses readline, calling venv misses _posixsubprocess, …)

But the install errors without any error messages make it hard to fix:
The necessary bits to build these optional modules were not found:
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
Failed to build these modules:
_asyncio             _bisect              _blake2           
_bz2                 _codecs_cn           _codecs_hk        
_codecs_iso2022      _codecs_jp           _codecs_kr        
_codecs_tw           _contextvars         _crypt            
_csv                 _ctypes              _curses           
_curses_panel        _datetime            _dbm              
_decimal             _elementtree         _gdbm             
_hashlib             _heapq               _json             
_lsprof              _lzma                _md5              
_multibytecodec      _multiprocessing     _opcode           
_pickle              _posixshmem          _posixsubprocess  
_queue               _random              _sha1             
_sha256              _sha3                _sha512           
_socket              _sqlite3             _ssl              
_statistics          _struct              _tkinter          
_typing              _uuid                _xxsubinterpreters
_zoneinfo            array                audioop           
binascii             cmath                fcntl             
grp                  math                 mmap              
ossaudiodev          pyexpat              readline          
resource             select               spwd              
syslog               termios              unicodedata       
xxlimited            xxlimited_35         zlib              
Could not build the ssl module!
Python requires a OpenSSL 1.1.1 or newer
I already checked that the installed openssl-devel version is 1.1.1d. Should be good enough.
Any ideas what could cause these or how I get decent error messages and not just this (which looks just fine to me):
building '_posixsubprocess' extension
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -fPIC -I./Include -I/home/joachim/Python-3.11.1/run/include -I. -I/usr/local/include -I/home/joachim/Python-3.11.1/Include -I/home/joachim/Python-3.11.1 -c /home/joachim/Python-3.11.1/Modules/_posixsubprocess.c -o build/temp.linux-x86_64-3.11/home/joachim/Python-3.11.1/Modules/_posixsubprocess.o
-pthread -shared build/temp.linux-x86_64-3.11/home/joachim/Python-3.11.1/Modules/_posixsubprocess.o -L/home/joachim/Python-3.11.1/run/lib -L/usr/local/lib -o build/lib.linux-x86_64-3.11/_posixsubprocess.cpython-311-x86_64-linux-gnu.so
building 'readline' extension
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -fPIC -I./Include -I/home/joachim/Python-3.11.1/run/include -I. -I/usr/local/include -I/home/joachim/Python-3.11.1/Include -I/home/joachim/Python-3.11.1 -c /home/joachim/Python-3.11.1/Modules/readline.c -o build/temp.linux-x86_64-3.11/home/joachim/Python-3.11.1/Modules/readline.o
-pthread -shared build/temp.linux-x86_64-3.11/home/joachim/Python-3.11.1/Modules/readline.o -L/usr/lib/termcap -L/home/joachim/Python-3.11.1/run/lib -L/usr/local/lib -lreadline -o build/lib.linux-x86_64-3.11/readline.cpython-311-x86_64-linux-gnu.so
              I tried with export LDFLAGS=–verbose and CFLAGS=–verbose. No error messages indicate failures
I found build/temp.linux-x86_64-3.11/home/joachim/Python-3.11.1/Modules/ which contains .o files of the missing modules (like_posixsubprocess.o, _ssl.o or readline.o) . So it looks like they compile just fine and it is a link issue?
              Read the output of configure it should tell you that is looking for libraries that it cannot find.

As you guessed you need to install the dev packages for the libs that cannot be found.

You may also need to add options to configure to tell it where to look for the header files and libs.
              Thanks,
the many failed modules to build were due to a temporary test that was not cleaned up by make clean. Doing make distclean now…
It is much better: no failed to build modules anymore. I don’t need nis and optimizations/test are disabled until the major problem is fixed:
The necessary bits to build these optional modules were not found:
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
The following modules found by detect_modules() in setup.py have not
been built, they are *disabled* by configure:
_ctypes_test         _testbuffer          _testcapi         
_testimportmultiple   _testinternalcapi    _testmultiphase   
_xxtestfuzz
No errors during make install, module libs are in run/lib64/python3.11/lib-dynload Still, I have the same symptoms:
job4:~/Python-3.11.1 > run/bin/python3
Python 3.11.1 (main, Feb  3 2023, 17:28:07) [GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
  File "/etc/pythonstart", line 7, in <module>
    import readline
ModuleNotFoundError: No module named 'readline'
I still have a lot of “… no” in the configure output, but I cannot figure out which (missing) libs are related to them. Also strange that the module libs seem to build anyways. Investigating…
 Joba 1:
I still have a lot of “… no” in the configure output, but I cannot figure out which (missing) libs are related to them. Also strange that the module libs seem to build anyways. Investigating…
Sometimes those are because of multiple options. A module depends on X, or Y, or Z, so it goes and tests to see if you have X installed… no; checks if you have Y installed… no; do you have Z installed… yes. So if the module worked, don’t worry too much about it.
The lack of readline usually means you’re missing a library called libreadline-dev or the equivalent in your package manager.
              Found by chance/desperation: internal modules were placed in $PWD/run/lib64 but python expects them in $PWD/run/lib
so, adding --libdir=$PWD/run/lib to configure parameters did the trick to get me a fully(?) working python.
This is the complete recipe
ver=3.11.1
wget -O- https://www.python.org/ftp/python/${ver}/Python-${ver}.tar.xz | tar xJf -
cd Python-${ver}
./configure --enable-optimizations --with-lto=full --prefix=$PWD/run --libdir=$PWD/run/lib
make -j8
make install
run/bin/python3 --version
Thank you.
 Joba 1:
the many failed modules to build were due to a temporary test that was not cleaned up by make clean. Doing make distclean now…
I recommend you write a script that does the build.

Start with rm -rf of your build dir.

Untar the sources.

Apply any patches you need.

Do the ./configure; make dance.
Then you know that there are no files left over from broken builds to trip you up.