Yes but the main thrust of his criticism is the fragility of the whole 'AI' stack, where every part of the stack is unprofitable, unstable and inter-reliant - if any one of those parts goes tits up - everybody else goes with them.
Also, a lot of AI 'users' arrive, search for an actual use case, find none, and then move on.
I would love to see an article about how people pronounce some of the strange names and acronyms we have ended up with. Quick survey? 'etc', 'gif', 'uefi', 'wget', 'cifs' ?
PDF is, without a doubt, one of the worst file formats ever produced and should really be destroyed with fire... That said, as long as you think of PDF as an image format it's less soul destroying to deal with.
PDF is good at what it's supposed to be good.
Parsing pdf to extract data is like using a rock as a hammer and a screw as a nail, if you try hard enough it'll eventually work but it was never intended to be used that way.
I think my fastener analogy would probably involve something more like trying to remove a screw that's been epoxied in. Or perhaps trying to do your own repairs on a Samsung phone.
It's not that the thing you're trying to do is stupid. It's probably entirely legitimate, and driven by a real need. It's just that the original designers of the thing you're trying to work on didn't give a damn about your ability to work on it.
Actually, parsing text data from a pdf is more like using the rock to unscrew a screw, in that it was not meant to be done that way at all. But yeah, the pdf was designed to provide a fixed-format document that could be displayed or printed with the same output regardless of the device used.
I'm not sure (I haven't thought about it a lot) that you could come up with a format that duplicates that function and is also easier to parse or edit.
It's pretty silly when you think about it. There's an underlying assumptions that you'll work with the data in the original format that you used to make the PDF.
QFT. PDF should really have been called “Print Description Format”. At heart it’s really just a long list of non-linear drawing instructions for plotting font glyphs; a sort of cut-down PostScript.
(And, yes, I have done automated text extraction on raw PDF, via Python’s pdfminer. Even with library support, it is super nasty and brittle, and very document specific. Makes DOCX/XLSX parsing seem a walk in the park.)
What’s really annoying is that the PDF format is also extensible, which allows additional capabilities such as user-editable forms (XFDF) and Accessibility support.
Accessibility makes text content available as honest-to-goodness actual text, which is precisely what you want when doing text extraction. What’s good for disabled humans is good for machines too; who knew?
i.e. PDF format already offers the solution you seek. Yet you could probably count on the fingers of one hand the PDF generators that write Accessible PDF as standard.
(As for who’s to blame for that, I leave others to join up the dots.)
PDF is great what it meant to be, a digital printed paper, with its pros (It will look exactly the same anywhere) and cons (Can't easily extract data from it or modify it).
Currently, there is no viable alternative if you want the pros but not the cons
For me, the biggest con of PDFs is that like physical books, the font family and size cannot be changed. This means you can't blow the text up without having to scroll horizontally to read each line or change the font to one you prefer for whatever reason. It boggles my mind that we accept throwing away the raw underlying text that forms a PDF. PDF is one step above a JPEG containing the same contents.
> Currently, there is no viable alternative if you want the pros but not the cons
I remember OpenXPS being much easier to work with. That might be due to cultural rather than structural differences, mind - fewer applications generate OpenXPS, so there's fewer applications to generate them in their own special snowflake ways.
This is the first time I heard of it. When I search for it I only find the Wikipedia article and 99 links to how to convert it to pdf.
The problem with this is that from an average person perspective it doesn't have the pros. There is no built-in or first-party app that can open this format on Mac and Linux.
More than 99% of the users only want to read or print it. It's hard to convince them to use an alternative format when it's way more difficult to do the only thing they want to do.
It's a Windows-thing, since W7, IIRC. It's ok now, but it has been buggy for years, and yes, who eats xps-files, so better it is, but it's not more useful.
We have to fill existing PDFs from a wide range of vendors and clients. Our approach is to raster all PDFs to 300DPI PNG images before doing anything with them.
Once you have something as a PNG (or any other format you can get into a Bitmap), throwing it against something like System.Drawing in .NET(core) is trivial. Once you are in this domain, you can do literally anything you want with that PDF. Barcodes, images, sideways text, html, OpenGL-rendered scenes, etc. It's the least stressful way I can imagine dealing with PDFs. For final delivery, we recombine the images into a PDF that simply has these as scaled 1:1 to the document. No one can tell the difference between source and destination PDF unless they look at the file size on disk.
This approach is non-ideal if minimal document size is a concern and you can't deal with the PNG bloat compared to native PDF. It is also problematic if you would like to perform text extraction. We use this technique for documents that are ultimately printed, emailed to customers, or submitted to long-term storage systems (which currently get populated with scanned content anyways).
You could probably reduce file size by generating your additions as a single PDF, and then combining that with the original 'form', using something like
There's itext7 (also for java). Not sure how it compares with other libraries, but it will parse text along with coordinates. You just need to write your own execution strategy to parse how you want.
From my experience, it seems to grab text just fine, the tricky part is identifying & grabbing what you want, and ignoring what you don't want... (for reasons mentioned in the article)
... and is much more difficult to extract text from than PDF, given that it's turing complete (hello halting problem) and doesn't even restrict your output to a particular bounding box.
I read ebooks on my Nintendo DSi for several years when I was in college; The low-resolution screen combined with my need for glasses (and dislike of wearing them) made reading PDF files unbearable. Later on I got a cheap android tablet and reading PDF files was easier, but still required constant panning and zooming. Today I use a more modern device (2013 Nexus 7 or 2014 NVidia Shield), and I still don't like PDF files. I usually open the PDF in word if possible, save it in another format, then convert to epub with calibre, and dump the other formats.
Epubs in comparison are easy, as all it takes is a single tap or button press to continue. When there's no DRM on the file (thanks HB, Baen) I read in FBReader with custom fonts, colors, and text size. It doesn't hurt any that the epub files I get are usually smaller than the PDF version of the same book.
Personally, I think the fact that Calibre's format converter has so many device templates for PDF conversion says a lot.
Well why not? I've used Delphi for years and also Java/JavaFX/Swing. There's not much reason to use Delphi anymore when Java IDEs can do the same things, and much more, and with docs that are just as good or better, all for free.
In Delphi I do desktop applications that perform real time low level device control, talk over USB and some specialized radio gizmos, DirectX graphics and multimedia processing, real time data presentation, bulk data processing, game like communications over UDP etc. etc. All at the same time from a different threads. Good luck doing it in Java. I did try doing low level stuff with it per client request and it felt like masochism. Java was created with the different things in mind.
The only thing that you seem to be right about is the low level device control stuff. Everything else is pretty much already easy enough to do in Java. The language was designed with multithreading in mind.
"...multithreading in mind..." - Easy and efficiency are 2 different things. Java synchronization primitives and it's ability to flip bits incur heavy performance penalty when compared to native ones. You might not care about it but I do.
You are also free to point me to a good DirectX framework in Java and/or the one that let's you write Directshow and Media Foundation filters and graphs as well.
Besides your logic reads like this: since language A does the same as language B and you happen to like language A then nobody needs B. Sorry but it works both ways.
When I was doing enterprise stuff Java provided more value to me in that particular situation. Windows desktop: sorry I'll choose Delphi any time.
Java synchronization primitives and it's ability to flip bits incur heavy performance penalty when compared to native ones
I think your knowledge on this might be a bit out of date. If you look at the Unsafe class (in old versions of Java) or the VarHandle class in newer versions, you can implement synchronisation and bit flipping with the same efficiency as Delphi, indeed it compiles down to optimal machine code.
You are also free to point me to a good DirectX framework in Java and/or the one that let's you write Directshow and Media Foundation filters and graphs as well.
Well, you can do this, albeit nobody really does because Java code tries culturally to be cross platform so using DirectX would be an odd thing for it to do. There are COM frameworks that let you write COM objects in Java. Whether you can do high performance image processing does depend on whether the JITC auto-vectorises though, at least until the vector API ships.
But you can for instance instantiate a JavaFX video player and then apply hw accelerated effects to it. Writing your own image filters would these days be best done in shader languages or lots of AVX anyway, so neither Delphi nor Java would really be appropriate for that.
Windows desktop: sorry I'll choose Delphi any time.
Yes, if you want to specifically write a Windows-only desktop app (that isn't WPF/WinUI/the new stuff I guess) and use lots of Win32 specific APIs, then Delphi is going to be better at that. On this we fully agree.
Still, these days people mostly want to write for at least Windows and macOS if not also Linux. And then you can write a good desktop app with JavaFX and the java packager tool (with native installers and the general appearance of a themed Windows app).
FWIW I used Delphi for years in the 1990s, even wrote a 3D video game with it in OpenGL. I've also written cross platform desktop apps with Java/JavaFX. So I have a lot of experience of both to compare them against. Delphi was fantastic at the time, and I still sometimes point out to people how the web platform fails to compare to even Delphi 2 + Windows 95 in many ways. I don't think we're really so far apart. But times have changed and Delphi wins only if you compare it to the web.
> Windows desktop: sorry I'll choose Delphi any time.
Let me quote you then.
> Besides your logic reads like this: since language A does the same as language B and you happen to like language A then nobody needs B. Sorry but it works both ways.
There are perfectly good enough tools for the Windows Desktop already and they are available directly from Microsoft.
Visual C is not such a good tool to write desktop application. .NET monstrocities with almost the same problems in regards to low level stuff, multimedia etc as Java when I can have single exe with no dependencies? Thanks but no thanks.
Also I do not really appreciate mentoring. Was doing just fine completely on my own for the last 20 years.
Positive reasons:
1) It runs out of the box without and Internet ocnnection.
2) The IDE is extremely good.
3) It teaches OOP and memory management (something often forgotten by today's programmers).
4) It is capable of giving very low level access to the host machine (like C).
5) You can even use (and teach) tricky things like pointers.
Last time i checked recent versions of Delphi have DRM that need internet connection. I even remember someone from the official newsgroups saying that he had to install a crack so that they can use it on foot in their laptop when they had no internet connection.
TBH, I found the low-level pointer stuff to be inaccessible. There is so much magic going on behind the scenes, and on the other hand it's hard to do pointer value arithmetic as well as pointer type arithmetic (you need to create aliases to no end). C is much more convenient for dealing with pointers, and it makes it much more obvious what happens from reading the code.
In Delphi culture, people say PChar instead of char* , and make tons of PFoo and PPFoo. And I think the reason is that some syntactical restrictions made Pascal easier to implement back in its time (last I checked there were still significant restrictions in Delphi)
