this post was submitted on 24 Aug 2024
38 points (95.2% liked)
Linux
47561 readers
676 users here now
From Wikipedia, the free encyclopedia
Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).
Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.
Rules
- Posts must be relevant to operating systems running the Linux kernel. GNU/Linux or otherwise.
- No misinformation
- No NSFW content
- No hate speech, bigotry, etc
Related Communities
Community icon by Alpár-Etele Méder, licensed under CC BY 3.0
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
If the files are literally duplicated (exact same bytes in the files, so matching md5sums) then maybe you could just delete the duplicates and maybe replace them with links.
Automatically sorting books by category isn't so easy. Is the metadata any good? Are there categories already? ISBN's? Even titles and authors? It starts to be kind of a project but you could possibly import MARC records (library metadata) which have some of thatinfo in them, if you can match up the books to library records. I expect that the openlibrary.org API still works but I haven't used it in ages.
I hope someone gives you a good answer, because I'd like one myself. My method has just been to do this stuff little by little. I would also recommend calibre web for interfacing instead of calibre. You can run both in docker, and access calibre on your server from whatever computer you happen to be on. I find centralizing collections makes the task of managing them at least more mentally manageable.
You might want to give an idea of the size of your library. What some people consider large, others might consider nothing much. If it is exceedingly large you're better off asking someplace with more data hoarders instead of a general Linux board.
I honestly don't know that there is one. What OP is looking for is effectively an AI librarian... this is literally a full-time job for some people. I'm sure OP doesn't have quite that many books, but the point remains
How many ebooks are you talking about (millions)? Is there just a question of finding duplicated files? That's easy with a shell script. For metadata, see if the books already have it since a lot do. After that, you can use fairly crude hacks as an initial pass at matching library records. There's code like that around already, try some web searches, maybe code4lib (library related programming) if that is still around. I saw your earlier comment before you deleted it and it was perfectly fine.