What was most surprising about this is not the fact that there is a group of people exploiting Stripe’s payments, but that the author had ChatGPT write a script to automatically handle payments processing, specifically for chargebacks. And based on the context in the article, the author sounds like they lacked the technical skill to write or validate these scripts themselves.
This author is jumping out of the frying pan and into the fire. ChatGPT is cool and all, but the fact that they’re trusting it to write critical code for handling their customers money speaks volumes. They’re incredulous at how they feel Stripe violated their trust in it to manage fraud, but then go ahead and blindly place it in another technology they don’t understand. The problem isn’t Stripe (though, yes, they should fix this), it’s the fact that they are just giving away trust and hoping for the best.
> the author had ChatGPT write a script to automatically handle payments processing, specifically for chargebacks
Feels like a mischaracterization tbh.
He had it make a script to go through and accept the chargebacks for these accounts, not handle payment processing or do anything to the chargebacks other than click "accept" essentially.
> And based on the context in the article, the author sounds like they lacked the technical skill to write or validate these scripts themselves.
I also don't really get where you're getting that from.
The author even said
> I reviewed all of the scripts carefully, and also never shared any customer data, IDs, or API keys. I think I saved at least a couple hours compared to hand-rolling these tools manually!
ChatGPT is not capable of writing production quality code. Many (most) companies have internal policies against deploying any code written by an LLM. The point isn’t to slow devs down, but to mitigate risk. This is especially important in the customer/payments stack. This is not the right place to “save a couple hours”. Maybe if this was for some one-off offline analysis, sure.
The fact that it works is insufficient proof that it was the right thing to do. Building a habit of relying on LLM generated code is an inherently risky practice, and ChatGPT will literally warn you against trusting its outputs. Sure, it lets you growth hack your way through sort term problems, but in the long term I’m not convinced this is responsible decision making at the current levels of LLM technology.
Or maybe I’m just a Luddite, stuck in my old ways.
Frankly speaking, probably the latter. I've been using Copilot for over a year now, and obviously it makes stupid mistakes, but it sped up my general coding speed. Now, I don't have much experience (maybe around 10ish years of programming professionally) in comparison to greybeards, but it works. Haven't used ChatGPT much, but as long as the user understands its shortcomings and reviews/refines its outputs, it's fine.
People who write code also make mistakes, yet we don't consider it "inherently risky practice". We just review others' code, tweak it, make it more appropriate for prod and voila. Same thing applies here.
>but as long as the user understands its shortcomings and reviews/refines its outputs, it's fine.
nice caveat doing a heckuvallot of heavy lifting. i understand that we're talking about coders and sort have this inferred impression that coders will have this understanding, but...that's an awfully broad brush you've used to paint over the simple fact that most people using LLMs (in general) are not understanding this.
Can we please get back to the actual topic is hand. The author clearly says that they took measures that’d seem to imply that they have an understanding of the shortcomings of ChatGPT.
You’re using this as a soapbox to cast holier than thou elitist aspersions on an imagined Everyone Else that isn’t as enlightened as you.
This exact “LLMs bad and here’s why!” thread of conversation is getting so old. The fact that it always has the same few talking points is evidence enough that those indulging themselves in it have been party to these conversations before. They know how it goes. And now it’s their turn to say the same old tired and in this case largely irrelevant things to sound smart and to pat themselves on the back.
> simple fact that most people using LLMs (in general) are not understanding this.
And why does that matter? Plenty of people "use JavaScript without fundamental understanding of the language's inner workings" but things are fine (not). My point is, people have always misunderstood the TOOLS that they use, but I don't see the same kinds of rejection before? Yes, people use LLMs thinking it is the end of programming but you can become way, way more productive as a programmer if you use it as a TOOL. The other day I used it to create a simple Python function to generate N distinct colors. It works, it seems to work, it even suggested using hue instead of rgb since its better or whatever (so I looked it up and it is for the eye) so I just used it. Should I spent a week going on a deep dive into the human eye perception and reading up articles on this?
Tons of production code, even in flows directly related to money (banks, payment portals etc) are hacked together from various stack overflow posts, not always with full understanding of the shortcomings, corner cases etc. Typical is some complex regex.
The latter, I'd have to suggest. GPT-4 generates code that is slightly better than the average junior programmer, which is to say, it is often confidently incorrect and needs review before committing, but either option remains a net productivity gain than no assistant at all.
"Your job will not be taken by an AI. Your job will be taken by someone assisted by an AI."
The process touched on in the article, with thorough review before commit by a human with in-depth experience of the language and APIs and the domain in question, is exactly how AI-generated code should be incorporated into a workflow. The earlier slander against the author's technical ability seems misguided and unsupportable.
I've used ChatGPT (GPT-4) to write production code.
As long as you keep the scope small ("Write some example code that calls $API in Python", "Make it asynchronous; so I can queue up n calls and execute them in parallel"), it generates perfectly good code that is easy to understand too.
This is outright false. I have used ChatGPT many times over the last couple months and I have caught it give me un-working code, unfinished code, and terribly buggy code. When you point this out it will say Oh sorry about that here is an updated version, and I've caught it give another bug, and another after that. If you are telling me the quality of code that ChatGPT gives you is high then it pains me to say but you must not provide high quality code yourself.
When you used google previous to chatgpt, did you force yourself to only allow yourself to use the “I’m feeling lucky” way of search along with having to use the result as your unadjusted production code. Did you never modify the code you came across?
Of course not, that’s ridiculous. You probably searched, read a few stackoverflow comments, found a relevant GitHub repo, a library for python/language of choice, and probably also a SAAS offering solely focused on the 3 lines of code you need. You quickly parsed all that and decided to modify some code in one of the SO comments for your needs. Next time, you looked passed half the junk and went straight to the first SO result and was able to tweak and use the result. The next time, it didn’t help but did help you write some inspired custom code for the problem, at least you knew what not to try.
My point being ai is useful. It’s not meant to be first result is final answer type solution, if that’s how you use it you will have issues.
How can you say that something is outright false if there is not fact/claim you can disprove. You’re responding to someone you don’t know and have no idea what they are working on.
I’m (not OP!) a cloud engineer but also work on a lot of FE (React) code for internal tools. ChatGPT has saved me countless hours (literally tens a month) writing super simple code that I am able to easily write up myself but typing it out just takes time. After month of using it I find myself still quite excited whenever cGPT saved me another hour. We also use Retool, but I find myself writing code ‘myself’ more often since cGPT launched.
No, I wouldn’t just copy paste production code handling PII, but prototyping or developing simple tools is sooooo much faster, for me.
Sure, it doesn't nail it 100% on the first prompt 100% of the time. Sometimes it takes a few prompts. It's no big deal. If you can't get it to write effective code, either you're working in a very niche area, or you haven't figured out how to use it properly.
Another reason someone can’t get it to write effective code is if they don’t know how to code or aren’t a very good programmer.
I use it a ton. Most of the time it’s very helpful, sometimes I can’t get it to write effective code. If the code it outputs doesn’t meet my standards, I just don’t use it. But I know what I’m looking for, and when ChatGPT generates it, if not only saves me a shitload of time, but more importantly it saves me a ton of mental energy that I can spend elsewhere. The biggest thing for me is that using ChatGPT helps my brain do fewer “context switches” between focus on high level business logic and low level implementation logic. By staying “high level” I’m able to accomplish more each day because I don’t get lost in the sauce as often.
I often have to “upgrade” the code myself with tests, better comments, modify the data structures a bit. Sometimes I tell ChatGPT to do this, sometimes I do it myself. But it’s been very helpful overall.
The big takeaway is that your output will only be as good as your own programming skill, regardless if you use ChatGPT or write it yourself.
I concur. It's just like any other tool, it's only as good as the person using it. I just can't understand the resistance of people in this field. I was a naysayer on a number of things like Docker when it first came out because it didn't solve any of my problems at the time. Then, k8s came out and Docker was a pivotal part of that solution, and k8s solves many problems.
ChatGPT writing code so you don't have to, I just can't conceptualize how that's not an instant win for just about everyone.
No, because junior devs usually improve over time.
I've tried Copilot and a few other AI codegen tools. Aside from producing overall low quality/nonworking code, the only times they seem to get better long-term are when a new update to the model comes out.
I should have been clear but ChatGPT was one of the "other AI codegen tools" I mentioned, especially as it's the one I used most recently. I tried it for a month or so but then canceled my subscription. I got some use out of it for answering questions for friends who were learning CS for the first time in languages I didn't know, but I didn't get much else from it which felt like it was high enough quality that it really saved me time or effort.
Edit:
And to contrast with junior developers: I find pairing with them something that makes me not only help me figure out the requirements of the things we're working on--which admittedly ChatGPT does do, but I think that's mostly by virtue of rubber ducking--but it helps me figure out approaches I wouldn't have thought of before, or encourages me to write more maintainable code by seeing when another person's eyes start glazing over.
Trying to claim that someone else’s personal experience is factually wrong? The internet teaches everyone great arguing quips, sure. But “outright false” actually MEANS something. Your comment is all emotion.
My current suggestion is to consider it the work of a just on-boarded intern. It will save you some time but you still need to walk thru the code to make sure it will work as intended.
First, it's worth noting the code in the blog post is not "production code," but rather one-off or periodically used scripts for accelerating manual business processes, with results that are easy to manually check.
But in regards to production code, I agree. When code is committed to a codebase, a human should review it. Assuming you trust your review process, it shouldn't matter whether the code submitted for review was written by a human or a language model. If it does make a difference, then your review process is already broken. It should catch bad code regardless of whether it was created by human or machine.
It's still worth knowing the source of commits, but only for context in understanding how it was generated. You know humans are likely to make certain classes of error, and you can learn to watch out for the blind spots of your teammates, just like you can learn the idiosyncrasies and weak points of GPT generated code.
Personally, I don't think we're quite at "ask GPT to commit directly to the repo," but we're getting close. The constant refrain of "try GPT-4" has become a trope, but the difference is immediately
noticeable. Whereas GPT-3.5 will make a mistake or two in every 50 line file, GPT-4 is capable of producing fully correct code that you can immediately run successfully. At the moment it works best for isolated prompts like "create a component to do X," or "write a script to do Y," but if you can provide it with the interface to call an external function, then suddenly that isolated code is just another part of an existing system.
As tooling improves for working collaboratively with large language models and providing them with realtime contextual feedback of code correctness (especially for statically analyzeble or type-checked languages), they will become increasingly indispensable to the workflow of productive developers. If you haven't used co-pilot yet, I encourage you to try it for at least a month. You'll develop an intuition for what it's capable of and will eventually wonder how you ever coded without it. Also make sure to try prompting GPT-4 to create functions, components or scripts. The results are truly surprising and exciting.
That's slow corporate thinking. I'm familiar with it but disagree because the efficiency gains are undeniable and it puts a business on a noncompetitive stance if they fail to change with the times.
ChatGPT and Copilot are like intern software devs who can produce code in seconds. They generate code that's usually close, but not always correct, at the savings of a great deal of your time of typing the whole thing vs. verifying correctness.
For critical and complex algorithms, it's not worth using ML coding assistants right now, but they will be in the future. It's obvious that that's where it's headed: massive efficiency gains for non-technical and barely technical people, and the decline in demand for software engineers, and with it, also a decline in software engineering salaries.
The main reason that the infosec folks have paused the use of LLM's to generate code is copyright concerns: who "owns" the code that's generated, the LLM, or, the company?
Realistically chatgpt isn't writing the financial code. Stripe did that already. Chatgpt is just reading snippets of Stripe's API examples for you and applying the code for a common use-case.
It's terrible, but not far removed from what's been already happening with "developers" copying and pasting StackOverflow comments into a text editor and making uninformed compiler-error-guided-edits until it runs then done!
The root of the problem here is people making production stuff who don't know wtf they're doing. If they turn to SO posts, LLMs, or "developers" on fiverr/upwork doing the same thing, is there really much of a difference? LLMs seem to mostly be tightening the loop of horror that's already been happening.
Just seems like programming will be joining the ranks of most tools. There will always be craftsmen, there will always be professionals, and then there will be the guy bolting together ice chests and garbage disposals to make margarita mixers on his patio or the kid with a duct tape exhaust rig on his Honda Civic.
I guess, to your point, it's only trouble if the margarita mixer guy is put in charge of something that matters? :D
(might be a bad example, I've known some fine engineers and mechanics that are absolutely margarita mixer guy, but hopefully my point is taken lol)
I'd love to know about the nirvana you've been in up till now, because working around code from numerous large companies the vast majority of it is the crappest ass crap straight from the crapper with no redeeming qualities, and it has been this way forever. I'm not saying their isn't good parts, there are general core routines that sheer need for them to be performant and non-data corrupting forced some Sr engineer to fix them.
This guy is operating a profitable business, creating value for customers, shipping features, and openly publishing details and learnings about the threats he mitigated. He used ChatGPT to generate scripts to help him throughout this process. I don't know if he's non-technical or if he just wanted to save some time, but frankly he should be commended for his hustle and get-shit-done attitude. These scripts were not determining life or death, or even making business critical decisions - they were filtering bulk data and making his life easier by producing results that are easily manually checkable, but save tons of time either coding the scripts or hiring a programmer to write them.
To me it reads like a great example of where ChatGPT is most useful: as a force multiplier for time-constrained entrepreneurs who have a specific goal and need specialized knowledge for short periods of time (e.g. to write a script). It's now basically free and instant to produce what would previously require a multi-week process of sourcing, hiring and communicating with contractors to write a script that leads to the same end result.
The kneejerk reaction to call this "surprising" or irresponsible, while understandable, gives major "get off my lawn" energy. This is the future and as coders we should support the increased self-sufficiency of non-technical people. If you want to adapt to the change then maybe think about how to improve the process for entrepreneurs of asking ChatGPT to write a script.
Article author here. I carefully reviewed and tested the ChatGPT scripts before executing them. It helped me save a lot of time manually writing these scripts!
I wouldn't say I lack technical expertise in this area, I'm just trying to use my time as efficiently as possible.
Genuinely curious: How much time would you say you saved prompting for and then carefully reviewing and testing those scripts for bugs, versus writing them yourself?
And for context what's the average line count we're talking about here? Tens of lines? Hundreds?
Glad to see you active here in the comments. Apologies if my comment comes off harshly, my intent is not to tear you down. I think there is a lot of gray space when it comes to using LLMs for generating code. Your usage here is certainly interesting, and I appreciate the additional context and discussion you’ve been providing.
No worries at all! I agree that there's probably lots of people blindly copying and running code from LLMs without any reflection. Just like it often happened with StackOverflow snippets before ChatGPT (to the point it became a meme). I'm definitely not one of them.
I don't know why I see this type of invalid speculation so often. The author already responded that they reviewed the script and didn't post any sensitive data, so won't add more to that.
I'd just state that tons of us use ChatGPT effectively and never blindly trust the outputs - for me ChatGPT is a starting point, not the final product. We're not all so daft as that lawyer who cut and pasted hallucinated case references into a legal brief without verifying them first.
100% agreed, this is how I always treat it and working on the problem from the article was not an exception from this rule. I share minimum input, and never trust the output blindly.
It gets 50-60% of work done, and a really good basis for me to work on it. Especially when working with one-off, end-to-end relatively short scripts.
This has been my primary use case as well (usually for writing some scripts or where I need to solve an operational task quickly), and ChatGPT has saved me a ton of time with those tasks.
I find it odd Chatgpt was mentioned at all. It was almost like an advertisement.
I have read post linked here similar to this one, but I can’t recall another instance in which the author abruptly said they relied on stackoverflow to code something unless the content was a meta commentary on coding and debugging itself.
I can empathize with the author. The first time you write some code collaboratively with GPT and it actually works, you feel a burning need to shout about it. Because it's one of those moments where something "clicks" and you suddenly feel like you've discovered fire. Once you figure out how to work with them, it makes you excited for the future and you can clearly see where LLMs will fit permanently
into your toolbelt. They're far from perfect now, and sometimes the time savings is a wash - you get instant specialized expertise that can produce code like a senior engineer, but you need to goad and coax it like it's a high maintenance intern. But the thinking power expended is still somehow lower - it's a new way of working with technology and deferring some of the grueling parts to the machine. This becomes especially obvious when the code requirements depend on an esoteric API or conventions that you'd normally need to spend time researching and manually enumerating.
Author here. My intention was to show that you can use it to help you get going quickly for a very practical, one-off, and self-contained use cases. As I mentioned in other comments already, I did not trust it blindly and did not share any sensitive data with it. Definitely not an ad!
I'm a huge LLM skeptic, but I'd disagree with you here.
I think using ChatGPT to write long-lived code for a serious application is a bad idea. But I think it's fine for somebody knowledgeable to use it for throwaway and first-draft stuff in areas that aren't their daily work.
Here's the author in question: [edit: wrong Piotr Mierzejewski in tech, see below]
He looks perfectly competent to me to evaluate the effects of some one-shot scripting code, so I think "giving away trust and hoping for the best" is a wild exaggeration of what actually went on.
They really wanted us to know they used ChatGPT too. It felt unnecessary how often they mentioned "I got ChatGPT to write a script that did this" like.. ok?
> I created a restricted key in Stripe with lowest possible permissions, and prompted ChatGPT to create a script to accept the chargebacks.
From my understanding, it also seems that the author submitted a Stripe API key alongside the prompt to create the scripts. This is pretty much a big security no no regardless of the permissions of the key.
This author is jumping out of the frying pan and into the fire. ChatGPT is cool and all, but the fact that they’re trusting it to write critical code for handling their customers money speaks volumes. They’re incredulous at how they feel Stripe violated their trust in it to manage fraud, but then go ahead and blindly place it in another technology they don’t understand. The problem isn’t Stripe (though, yes, they should fix this), it’s the fact that they are just giving away trust and hoping for the best.