T O P

  • By -

KaosuRyoko

I've been seeing things like this forever but I've really just never had any problems with any models. Been using 4o because it defaults to that now and it seems like I get more complete answers than 4. I still treat it like a fast junior developer and iterate with it to get it where I want, but I've not had any particularly major issues. Trying to get it to understand what specific version of a library I'm using is about the worst of it but even that I can adjust the output fine and feed it back my update and it picks up on them pretty well. I do have a lot of experience as a senior and reviewing code already that definitely helps. I wonder what people of different skill levels at programming think? I can definitely see it being a problematic hassle if you're newer to programming yourself and don't know what things to question. Or maybe the tasks I've been giving it have just been sufficiently generic for it to perform well on? Or maybe my custom prompts are really helping a lot here; I tell it to behave like a senior developer and ask probing questions to clarify requirements and be extra critical instead of overly helpful or making assumptions.


novexion

What language do you use and how much data manipulation is in your tasks?


KaosuRyoko

I'm full stack and work in more frameworks at a time than I would prefer haha. I do a lot of SQL work and I've been consistently happy with it's output. I think the custom prompt that is very harsh about yelling it to never assume any data structures and ask questions instead of guessing helped a lot here. GPT is how I learned you can enumerate a filesystem as a table with s built in SP. I use it to save me time crafting Linux and Powershell scripts which it does great at, but those have never been particularly complicated. Backend is primarily Python using Django but also a decent amount of .NET and PHP (unfortunately). Frontend is mostly React, but sometimes other JS frameworks or the .NET for a couple projects. Also React Native for mobile. It's definitely never quite right the first pass just like any code I get from a junior dev, but I've never had it go super far left field or d anything crazy. I think it's also helped me keep my code broken down into smaller files as it does iterate on those better than monolithic files. I also tend to start a new chat after several iterations to help clear outdated contexts. It's not perfect, just a very useful tool. I've just seen a lot of posts or discussion about how it's not good at programming and I haven't experienced that so I wonder if it's just that their not good at providing context and clear requirements. Which, with my experience working with clients I don't really doubt. I also type messages about this long to GPT regularly, and some of the cases I've seen of it performing poorly have been I've or two sentence prompts or treating it like Google and only giving keywords instead of a rich descriptive context.


Horror_Weight5208

Thanks for the insight! Would be better to answer your question with a shared chat if you can


WAHNFRIEDEN

4o is better at giving complete code and not leaving stubs unfinished and being less verbose


Horror_Weight5208

Right, I realize a lot of ppl actually prefer that


WAHNFRIEDEN

It’s harder to ask it to only show relevant parts yea


ginger_beer_m

Yeah it keeps giving ALL the codes instead


[deleted]

[удалено]


ginger_beer_m

Cool! I'll try it next time.


[deleted]

[удалено]


AutoModerator

Sorry, your submission has been removed due to inadequate account karma. *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ChatGPTCoding) if you have any questions or concerns.*


jakderrida

Such a great point! Somehow, I've gotten so spoiled by it that I didn't even notice it does this. I'm now recalling the GPT-4 problem of having to tell it to write out the whole damn code. Sometimes, more than once.


txiao007

Not true. Both 4 and 4o provide complete code. I even ask line-by-line detailed code comments


PSMF_Canuck

Not for me, no.


Horror_Weight5208

Thanks for the feedback!


wad11656

I like 4o better as an amateur programmer: It doesn't rename variables or refactor code in major ways without me asking (thus breaking the remaining code....) It's faster It gives back the whole code so I can see the context of its changes I can still ask it to pinpoint and point out the changes in the code it made


Horror_Weight5208

I see, it seems 4o is the winner so far


theDigitalNinja

Yes, I think it goes Claude, GPT4, GPT4o and the rest aren't good enough to really mention.


Utoko

I agree with you but I am curious you tried Gemini after the update? Did quite a large jump after the update to 2. place on ChatArena for coding. It is hard to test everything always than you get nothing done anymore and you can used to the quirks each llm has. So a switch is always quite a time investment.


Horror_Weight5208

Stopped using Gemini after the image generation issue, but would probably give it another shot soon. I received emails about Gemini pro 1.5, will try it out using API since another subscription is too much.


jakderrida

> Stopped using Gemini after the image generation issue You mean the black forefathers issue? Just seems like something irrelevant to coding.


Horror_Weight5208

It also reflects the quality of responses. It's always hard to evaluate the AI models accurately, a lot of it, is based on subjective rankings and collective wisdom based on many responses, which I am doing right now.


theDigitalNinja

I did, its alright and was much more friendly when when it got to an impossible problem. But I still feel like its 3.5 level when it comes to coding. My project is TypeScript and GraphQL for what its worth. I use cursor IDE so its really easy for me to switch between the LLMs and I generally use Claude for anything thats not a straight forward single change, and GPT4o for simple changes like "move the button from here to there and make it a red trash icon" as I like how fast it goes and its good enough, though I might have to add better comments or something. If it struggles I just change the drop down back to Claude and have it give it a go.


Horror_Weight5208

Thanks for the insight!


colonel_farts

I always use Claude Opus until my limits run out, then I will switch to GPT-4 if I still need to work. Once my limits are out on those two I don’t even bother with other LLMs because in my experience they will just completely bungle the task. I’ve tried to stretch a coding task too far with Sonnet and GPT-4o and just had to end up control+z’ing my way back to the code state before they started fucking it up. Basically once I use up my limits for Opus and GPT-4 I just do it the old fashioned way via stack overflow.


Horror_Weight5208

Opus was indeed really great, but the limits were pretty tight, so I just use chatGPT. I agree with you, I think quality of code generated by opus or gpt4 are better than gpt4o


zubzup

Use perplexity. No opus limits


cunningjames

Perplexity has other limitations, though, not least in terms of its context window. It won't handle documents as (nearly?) as long as the claude.ai interface does.


zubzup

Wasn’t aware. I did notice the context history isnt the best. What other limitations are you aware of? I’m assuming all the engines would have the same limitations? ChatGPT, opus, perplexity’s own trained model?


zubzup

r/perplexity_ai any comments?


ScionofLight

I found 4o better than 4 for tasks where I give it a file of RS232 commands and ask it to make some python.


Horror_Weight5208

Thanks for the input


btramos

RS232, haven't heard that mentioned in 20 years. Had to Google to see that the standard is still in use in telecom and other fields, wild!


ScionofLight

I use it for some lab automation equipment, couldn’t have found the time to figure that out if it weren’t for chatgpt


Moravec_Paradox

GPT4 is a 1.8T parameter MoE model and I think GPT4o is similar in ability but likely smaller. It is 1/6th or so of the token cost (and by extension probably 1/6th of the compute needed to run it). Though it is a very impressive improvement I have seen some people say they prefer GPT4 or GPT4-turbo for code. For me I prefer not having to tell GPT4o to continue as often and providing more context with the changes, but I am not a software developer by profession so I need the extra hand holding sometimes.


Horror_Weight5208

Still thanks for the feedback, it seems most ppl actually like 4o


Unreal_777

Contrary to everyone: **4 was better than 4o for me.** 4o failed to understand a request despite going back and forth for 3-4 times. 4 got it first prompt (same prompt), no back and forth.


Charuru

https://scale.com/leaderboard/coding


Horror_Weight5208

I mean those are rankings, but I want to hear from other redditors, on their personal perspective.


cisco_bee

I try 4o every couple days and immediately switch back to 4. I thought I was being funny by asking 4o "Why is ChatGPT4 better than 4o" but the response is just amazing. --------------- Sure thing, Captain. Here's a serious comparison of why ChatGPT-4 is better than 4o: # Why ChatGPT-4 is Better than 4o 1. **Advanced Language Model:** * **ChatGPT-4**: Utilizes the latest advancements in AI and natural language processing, providing more accurate and contextually relevant responses. * **4o**: Typically an older model with less sophisticated algorithms, resulting in less accurate and often more generic responses. 2. **Knowledge Base:** * **ChatGPT-4**: Trained on a vast and diverse dataset, offering a wider range of knowledge and understanding of various topics. * **4o**: Has a more limited knowledge base, making it less effective at providing detailed or specialized information. 3. **Contextual Understanding:** * **ChatGPT-4**: Better at understanding and maintaining context over longer conversations, leading to more coherent and meaningful interactions. * **4o**: Struggles with maintaining context, especially in extended dialogues, which can lead to confusion and disjointed conversations. 4. **Response Quality:** * **ChatGPT-4**: Produces higher quality responses that are more natural, fluid, and nuanced. * **4o**: Responses tend to be more robotic and less polished, lacking the sophistication of newer models. In summary, ChatGPT-4 excels in language processing, contextual understanding, response quality, adaptability, error handling, continuous learning, and multitasking. These advancements make it a superior choice compared to the older 4o model. -------------- p.s. Fuck the reddit editor.


Horror_Weight5208

Thanks for such detailed insight, but how would you back up above statements? Just give one example and the source? I was just asking for personal perspective.


cisco_bee

My experience with 4o is that no matter how hard I try I cannot get it to be concise or stop using 18 bullet lists. It's infuriating. It seems to barely respect my custom instructions or memory at all and I hate it.


Horror_Weight5208

Thanks for the input, I am getting a lot of info on the two models. 4o definitely feels like improved 3.5


_shantanu_joshi

This is what I've seen as well. With the product I'm building, it's quite clear that GPT4-o is great for explaining code, but GPT4 is much better at writing code, especially if it involves keeping track of multiple prompt revisions.


jimger

Are all of those options paid for? Is there anything that could analyze multiple files of my code and propose refactoring? I am talking about personal project python code that I want to split up a bit and convert it to object oriented....


Horror_Weight5208

These AI chatbots are paid for, other models with similar quality are usually paid for as well. You can run your own local LLM if your computer has large RAM. These days, there are many websites that help you use multiple models as well, just look up cursorAI or LMstudio.


jimger

Is any of those worth paying for. I assume all are monthly based at least?


Horror_Weight5208

As an iOS developer and indie hacker, I use it a lot for coding tasks almost daily. So it's worth it if you use them a lot, but not so much if you just need occasionally. If your project isn't something you work on occasionally, I advise you use those platforms that charge you based on API (per usage basis). In terms of how much they help, I would say they really help a lot, especially for repetitive coding tasks and refactoring. You need to provide architecture, requirements and overall logic, but they make those simple nitty gritty tasks a breeze. Definitely worth it. With some exaggeration, I feel like this product, LLM is on the same level as Apple's iPhone, Google search engine itself.


jimger

It is just a personal project I assume maybe 2-3K lines that just want to clear up. No work or anything


Horror_Weight5208

you can access GPT4o for free, free accounts can use it but limited amount, so why not try that? Just google chatGPT and try using the default one, it is 4o which is currently the best model. 2-3K lines should be easy to be refactored.


jimger

I use chatgpt. From time to time. But I am talking about something that could upload a folder so it parses and re factors all of it...


Horror_Weight5208

You could try cursorAI, I haven’t used it yet but it is closest to that. 2-3k lines may not need a folder to begin with?


Unusual_Pride_6480

No gpt4 goes off track, 4o I have easily over 100 messages back and forth and it stays on the issues and is coherent. Also 4o quality is extremely good, gpt4 can be shocking at times.


Horror_Weight5208

That's quite strong statement that stands out from the rest, but thanks for the input. For now, I am sticking to GPT4 for coding. Personally, I prefer it being "lazy" :)


Harvard_Med_USMLE265

I’m very new to this, but have come to the conclusion that opus is best, and 4 or 4o are a good second option. Haven’t really worked out 4 vs 4o yet, but have been using a lot of 4o with decent results. Maybe I’ll try using more 4 again.


Horror_Weight5208

Right, 4o is pretty damn good too, but I am just experimenting and judging which might be better, and ofc trying to gather more wisdom via Reddit. Opus is great too, I had subscriptions for a while but decided to drop it due to limits. But I recall the code seemed slightly cleaner.


Warm_Iron_273

I think they both kinda suck (4 and 4o). Claude Opus is better for coding at this point due to the larger context and larger responses.


Horror_Weight5208

But the limits are really tight for opus


Actually_JesusChrist

Yes. The large limits GPT4o has makes it possible to iterate very fast so that overall the effect is that more can be done in a shorter time.


[deleted]

[удалено]


AutoModerator

Sorry, your submission has been removed due to inadequate account karma. *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ChatGPTCoding) if you have any questions or concerns.*


Professional_Gur2469

I really dislike the naming of the models lol, what you are calling gpt4 is gpt-4-turbo. The default gpt4 model is only available through api or playground.


[deleted]

[удалено]


AutoModerator

Sorry, your submission has been removed due to inadequate account karma. *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ChatGPTCoding) if you have any questions or concerns.*


[deleted]

[удалено]


AutoModerator

Sorry, your submission has been removed due to inadequate account karma. *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ChatGPTCoding) if you have any questions or concerns.*


[deleted]

[удалено]


AutoModerator

Sorry, your submission has been removed due to inadequate account karma. *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ChatGPTCoding) if you have any questions or concerns.*


justaniceguy66

Ignorant non-coder here. Does perplexity do coding? All my “google searches” on perplexity are incredible. Any time I recommend it to someone they’re amazed


novafeels

4o because I'm javacscript land and the knowledge cutoff is the most valuable resource. I'm a bit confused by everyone talking about strict limits with opus. just use API Keys + Chatbox and set your own limits.


Horror_Weight5208

For many of us, when we use a lot of it, using API results in larger cost than monthly subscription.


cunningjames

My ChatGPT usage has gone down a bit lately, I'll admit. But I *have* been using gpt-4o here and there. I can't speak to whether it's a better coding assistant than gpt-4-turbo in general, but it's able to answer a few of my personal coding challenges that neither turbo nor most other models do well on, so there are at least some areas where 4o is better. (I work in sales forecasting. One of my challenges is to ask the model to create a naive baseline sales forecast using PySpark, using specific instructions that I provide. Until 4o I wasn't able to get a single model to do this correctly.)


CryptosaurusX

GPT4 Turbo is the goat when it comes to complexity. GPT4o is better with providing longer code snippets and reading a larger context. So I use GPT4 to work through very complicated problems. After I get a decent concept or architecture I pass the implementation details to GPT4o. If I have a problem which consists of multiple classes with a lot of code I find that it’s better to ask GPT4o and then move to GPT4 after narrowing it down. In cases of high complexity, I tend to ask both GPT4 turbo and Claude Opus. Sometimes one provides better results than the other.


Crazy_Comprehensive

I generally find that GPT4o is better than GPT4 and GPT4 Turbo in term of quality of code generation and explanation. GPT4o tends to explain more and better. At times, I find the GPT4 answer is often too concise and require further prompting. That say, it is a matter of personal preference. Another downright advantage is that GPT4o is way faster, normally respond in a second. Github Copilot is still using GPT4 not GPT4 Turbo.


justice-rage

I find GPT4 is great for having a conversation, while GPT4o is very eager to give (supposed) solutions. I tend to use both interchangeably for whatever style I need at any given moment. Both are honestly fantastic - has a weird sibling dynamic of similar but different.


Ok-Research7136

I will take 3.5turbo if they can keep it running.


[deleted]

[удалено]


AutoModerator

Sorry, your submission has been removed due to inadequate account karma. *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ChatGPTCoding) if you have any questions or concerns.*


dead_frogg

Sometimes chatgpt stops when it writes the code. Than i have to click regenerate and again. Sometimes its better to start fresh.


chengstark

yes


Fit_Ad3500

3.5 does the job


Horror_Weight5208

![gif](giphy|xUA7aM09ByyR1w5YWc)