Install Python 3.12 in CentOS 7 with SCL

In the previous post, I documented my attempt to install Python 3.12 on my CentOS 7, which ended in failure due to compilation errors. After an extensive search, I identified the issue as an outdated version of gcc, with CentOS 7 using gcc 4.8.5 while the latest version is gcc 13. Upgrading gcc to the latest version in CentOS proved challenging, but Software Collections (SCL) offered a potential solution. Given the considerable time investment required, I didn’t proceed with the SCL installation and experimentation. This post is motivated by that decision. I will explore SCL in this post, aiming to successfully run Python 3.12 with the assistance of SCL.

Read more

Get File List of Dataset from Huggingface without Downloading Data

Hugging Face is both a platform and a community focused on machine learning, particularly in the areas of natural language processing (NLP), computer vision, and audio processing. It is often referred to as the “GitHub of AI,” serving as the primary platform for hosting a vast majority of models and datasets in the AI community. It also furnishes tools that simplify the process of downloading models and datasets. With the Hugging Face Python libraries, achieving this task requires just a few lines of code. For instance, utilizing the huggingface_hub to load models and the datasets library to load datasets.

Read more

Install Python 3.12 on CentOS 7

I’m currently immersed in learning AIGC, and Python is an essential tool for almost every aspect of it. In my attempt to set up Python on CentOS, I encountered installation challenges with both Python 3.12 and Python 3.11, leading to installation failures. Both of them gave me following error:

Python runtime state: initialized
Traceback (most recent call last):
  File "/local/usr/local/src/python/Python-3.11.0/Lib/site.py", line 73, in <module>
    import os
  File "/local/usr/local/src/python/Python-3.11.0/Lib/os.py", line 29, in <module>
    from _collections_abc import _check_methods
SystemError: <built-in function compile> returned NULL without setting an exception
make[1]: *** [Python/frozen_modules/abc.h] Error 1
make[1]: Leaving directory `/local/usr/local/src/python/Python-3.11.0'
make: *** [profile-opt] Error 2
Read more

Running BitTorrent from the Command Line on CentOS

I’m on a mission to gather a massive 100,000 hours of voice data for training an advanced AI. But here’s a problem – grabbing just the voice part from the internet is tricky. So, I got a plan: why not download a bunch of movies and shows and then extract the voice data? That’s where BitTorrent comes into play – a powerful method for sharing files. Now, normally people click buttons in a bit torrent client to get files, but with a colossal 40,000 movies on the agenda, that’s just too slow. That’s why I opted for a more technical approach using bit torrent in the command line. It’s like giving the computer direct instructions to efficiently fetch all those movies for me.

Read more