I've got a monitor dedicated 100% of the time to ChatGPT, and I interact with it non stop during the flow of technical scenarios and troubleshooting situations that flow into me - working in areas that I have the slimmest of backgrounds in, and shutting down, root causing, and remediating issues that have been blocking others.

I've essentially got 15-20 high-priced world-class consultants in every field that I chose to pull from, working at my beck and call, for $20 a month? I would pay $200/month in a heartbeat out of my own pocket, and I probably would ask the company to pay ~$2,000/month for my workflow.

I think if they never released another product, and they just managed to penetrate with their existing offering, they are easily a $100B+ company once they nail down how to monetize.

The difference between LLMs and Crypto is I can point to roughly 200-300 objective solutions over the last 9 months where ChatGPT resolved an issue and delivered clear value for me alone. And, over time, as you learn how to control for hallucinations, and manage your query patterns a bit more - the value has continued to increase.

That same multiple-times-a-day high value persistent experiences were never a part of my crypto experience.

I keep reading responses like yours, but I haven't seen any specific examples of problems being solved, so it all sounds very abstract. In my interactions with ChatGPT, it felt like just interacting with a search engine. There were zero continuations between questions and responses, and nearly 100% of the responses contained incorrect information.

Edit 1: As an example, I just asked ChatGPT to implement the Warren Abstract Machine for me. It gave me two different implementations, both with caveats that they are simple examples and not the whole thing, and neither implementation even type checked. It feels just like reading someone's homework where they copied off of someone else's work and had no idea what's going on. I don't see the point in this if it's just going to give me some high-level idea, which I already have, and an implementation that isn't remotely complete, much less one that doesn't even run. "Additionally, you'd need to ... handle backtracking". You don't say, ChatGPT?

Edit 2: I've kept on asking it to implement things I already know how to implement, even things that it probably has my code (being it was on GitHub) in its training data, and it keeps giving me code that doesn't even typecheck. ChatGPT is just doing what I've always imagined it doing: it's just doing a statistical merge. It has zero concept of anything I'm asking or it's saying.

Edit 3: I asked it something regarding WebGPU. It gave the typical, oh that's complex and "Here's a simplified example using GLFW and WebGPU through the F# WebGPU bindings provided by FableGL". But FableGL isn't a thing that exists, and even if it did, it wouldn't have anything to do with WebGPU which has nothing to do with OpenGL. And it imported Fable.Import.WebGPU and Fable.Import.GLFW, neither of which exist.

I mean, this is literally all smoke and mirrors. It boggles my mind when I hear people say they successfully use it every day. I haven't ever got it to tell me anything remotely correct.

I've been doing dev for ~20 years now, most of the work out there is essentially plumbing events/databases/REST APIs/regexes/JSON/configs/ web UIs in existing and well established frameworks/tools to solve business problems.

You likely work more R&D style environments, which again, probably represent a very low % of devs out there

Grandparent specifically said they find ChatGPT useful for things outside their wheelhouse:

>> [...]troubleshooting situations that flow into me - working in areas that I have the slimmest of backgrounds in

It sounds to me like it may not be safe to use ChatGPT if you can't check its work.

- Paste a bunch of log code about which I have no idea. Ask to identify and explain the problem.

- Wireshark / dmesg / OpenWrt configuration pasting, ask to fix the problem. For instance, I fixed a Wi-Fi issue in a heterogeneous setup, which turned out to be caused by a stray DHCPv6 server.

- Paste C code, along with an error log, and ask to fix the problem.

- Paste my program and a sample. Ask to extend my program.

- Proofread and format Markdown nicely.

- Paste government letters, asking for a response that includes <what I want>.

- Paste a chat log and obtain documentation.

- Paste a tax declaration, and ask to check for consistency.

- Paste my code and ask for critique.

When discussing versions, people often confuse versions 3.5 and 4. I am always referring to version 4.

Keep in mind that ChatGPT has been seriously and intentionally downgraded since March.

This debate frequently leaves me wondering if I'm encountering a coordinated effort by bots. The examples I listed above come very naturally to me. I can't understand why people don't try to paste whatever they're working on and check the results. If it's too complex, asking it to critique instead of create, because that's easier. It feels as though there's an effort to shape public opinion into viewing these tools as "immature" and suitable only for edge cases.

Basically this - even in a highly specialised field you would still find value in it if you're providing enough context and describing the problem properly.

People don't want to hear that they're poor communicators - it's much easier to blame the tech.

It's just hard to even know what to say to this. So you are implying that the problem lies with the human, who holds years of experience in education, problem solving, communication, etc., and not the "AI", which has no experience in any of that and has no real thought processes or problem solving or reasoning abilities? And people say ChatGPT is over-hyped!

I've had some chatlogs where people are not able to articulate even one clear sentence. They were fine telling me what they want, but not able to type the same sentence

This is so weird and seems to be a common thread. You, and others, seem to be saying that ChatGPT is not useful if you're too dumb or too smart. It just seems both tautological and strange.

My communication skills are fine. ChatGPT is just not useful for me, period. I've tried taking what people have mentioned here, trying to have a so-called dialogue with it when things it suggests, for much more simple things, aren't quite correct. It keeps suggesting all these things, but it just feels like an interactive StackOverflow, except all the suggestions are wrong or don't work.

I am switching to teaching / consulting in AI. Chatbots are a part of that. I've reviewed some chatlogs, and some people get abyssimal results, which are entirely explainable and are 100% their fault.

I found it getting worse for a bit but lately it's been really good. I would say it's being constantly updated, not necessarily getting worse.

It's not able to output complete code.

It always adds comments, even when I ask it not to.

It doesn't follow instructions in general.

In complex issues it has less "memory", that is it's not able to observe multiple constraints at the same time. The limit was much higher before.

Sometimes you have to ask it in very specific ways to achieve what you're wanting - it is definitely limited in some areas though.

https://chat.openai.com/share/64dc6384-1fc4-4506-9280-98dc8a...

Just take any code which is < 3 pages long. Ask it to optimize a function. Or review the code. It should give good results

EDIT: I had some deprecation warnings in python recently, which it quickly fixed. Googled that on a different computer and found https://stackoverflow.com/questions/76689364/how-to-fix-depr...

This is the transcript. Forgive my dumb last question, I didn't understand the code, what it did worked well.

---

User: Please fix the deprecation warning on this code: for i in range(0,3): df.loc[df['date'] < "2023-06-01", 'date'] = df['date'] + pd.DateOffset(years=1)

ChatGPT: The code you've provided updates the 'date' column in a DataFrame df by adding one year to each date that is earlier than June 1, 2023. However, this operation is performed multiple times due to the loop, which is likely not intended and could be the source of inefficiency or unexpected results.

To fix this, you should update the dates in a vectorized way without using a loop. Here's the corrected code:

  import pandas as pd
  # Sample DataFrame creation for demonstration (you should use your own DataFrame 'df')
  # df = pd.DataFrame({'date': pd.to_datetime(['2022-05-01', '2023-07-01', '2024-08-01'])})

  # Correcting the date manipulation
  df.loc[df['date'] < "2023-06-01", 'date'] = df.loc[df['date'] < "2023-06-01", 'date'] + pd.DateOffset(years=1)
  *This code will effectively update all the dates that are earlier than June 1, 2023, by adding one year to them, and it does this in a single pass without a loop. This should also avoid any deprecation warnings and improve the efficiency of your code.*

ChatGPT: Certainly! Below is the code block with both the original version and the improved version. The improved version eliminates the unnecessary loop and applies the date modification in a more efficient manner.

  import pandas as pd

  # Original code
  df = pd.DataFrame({'date': pd.to_datetime(['2022-05-01', '2023-07-01', '2024-08-01'])})
  for i in range(0,3):
      df.loc[df['date'] < "2023-06-01", 'date'] = df['date'] + pd.DateOffset(years=1)

  # Print the result of the original code
  print("Original Code Result:")
  print(df)

  # Improved code
  df = pd.DataFrame({'date': pd.to_datetime(['2022-05-01', '2023-07-01', '2024-08-01'])})
  df.loc[df['date'] < "2023-06-01", 'date'] = df.loc[df['date'] < "2023-06-01", 'date'] + pd.DateOffset(years=1)

  # Print the result of the improved code
  print("\nImproved Code Result:")
  print(df)
  *This code first runs the original version, prints the result, then runs the improved version, and prints that result for comparison. Note that the DataFrame df is reinitialized before running the improved version to ensure a fair comparison.*

Code: >>> import pandas as pd >>> # Original code >>> df = pd.DataFrame({'date': pd.to_datetime(['2022-05-01', '2023-07-01', '2024-08-01'])}) >>> for i in range(0,3): ... df.loc[df['date'] < "2023-06-01", 'date'] = df['date'] + pd.DateOffset(years=1) ... >>> # Print the result of the original code >>> print("Original Code Result:") Original Code Result: >>> print(df) date 0 2024-05-01 1 2023-07-01 2 2024-08-01 >>> >>> # Improved code >>> df = pd.DataFrame({'date': pd.to_datetime(['2022-05-01', '2023-07-01', '2024-08-01'])}) >>> df.loc[df['date'] < "2023-06-01", 'date'] = df.loc[df['date'] < "2023-06-01", 'date'] + pd.DateOffset(years=1) >>> >>> # Print the result of the improved code >>> print("\nImproved Code Result:") Improved Code Result: >>> print(df) date 0 2023-05-01 1 2023-07-01 2 2024-08-01

ChatGPT: You are correct. The two code versions do not produce the same result. The original code with the loop increments the date by one year for each iteration of the loop, but only for dates before "2023-06-01". This results in multiple increments for dates that are initially before this threshold.

In contrast, the improved code without the loop increments the date by one year only once, making it more efficient and potentially more aligned with what one might expect logically. However, if the intent of the original code was indeed to increment certain dates multiple times, the "improved" version would not be equivalent.

If the goal is to replicate the logic of the original code but without the deprecation warning and with improved efficiency, we need to understand the exact behavior intended with the loop. If the intent was to increment dates before "2023-06-01" by multiple years based on some condition, the improved version should be adjusted accordingly. ---

---

Stay classy. The rest of your comment would have been much better without this ad hominem.

I once spent an hour researching the responses on reddit and hackernews. Turned out if these profiles are fake, then they are really good. F.e. age is >5 years, they argue about sport teams and seem really human.

Also, although I haven't seen it before, the Warren Abstract Machine seems like too big of a job for GPT-4. It excels at smaller tasks such as "Convert this C code to idiomatic Rust", or "Write a SQL query that grabs records from X, joins with Y, filters by Z". You might need to make small adjustments manually or by saying "Rewrite it with this in mind".

One really neat trick it did recently is that I uploaded a chart png I found on the internet, and asked it to write Seaborn code that matched the chart's style. It took ~3 follow-up tweak prompts, but then spat out code that worked. It also handles things like "How do I adjust this label that's positioned in chart coordinates over by 10 pixels?", which is an absolute pain to figure out from the docs.

Especially since you're getting answers noting the complexity, I think you're just asking too much of it for now. Try smaller tasks for now, and wait a while to try the big ones again.

So in order to properly evaluate a service, we need to already be subscribed to it in the first place?

A mundane one: I have a list of lat-longs mapped to region codes, and I needed write a query to find the area of the convex hull of each region code. I knew how to do it in code, but I wanted a SQL answer. ChatGPT gave me the correct Redshift SQL.

A more involved one: I needed to maximize match x people y groups, where people can rank their choices up to 3, but as soon as they accept the match, they drop out of the pool. Plus a bunch of different constraints. Sounds like a stable marriage problem. ChatGPT proposed the Gale-Shapley algorithm, which was the exact one I was looking for.

- Helping to refactor SQL

- writing jq commands (I simply cannot)

- writing shell code (it happens just infrequently enough that I can't justify spending the time to get good)

- brainstorming names or puns. Word association is easier with a second person (or an LLM)

- figuring out why my AWS CLI commands aren't doing what I'm expecting them to be doing

- asking for reasons why a piece of code wouldn't work

I can competently do all of these things on my own if I havw to, but now I don't have to do them on my own so my life is easier.

I get a half decent expert to solve problems I don't have the patience or experience to solve myself. That's literally the definition of what a consultant does. What you call "trivial" is what lots of other people describe what consultants largely do.

I’ve also used it quite a lot for advice as I was learning project management and moving away from individual contributions.

So I think as a consultant it has value, though I think “world class” might be a bit hyperbolic, since I still need to validate its output.

Can you expand on this? This is what I'd like to use it for, but I don't ever have any experience remotely close to helpful. The typical response is just something like "that's complicated but here's some gibberish that tries to show a simple example but is still wrong in ways it's not useful at all".

Do you have a specific example?

The code snippets tend to be bloated, but no worse than an intern would produce. A quick refactor, adding edge case handling and the code is ready to test and commit.

I think it's worth stepping back here and re-examining the hurdles your setting for your own understanding.

The essential question here is: is ChatGPT useful to people. What you seem to be implying with your question is: I will not use ChatGPT unless it can solve problems with X level of difficulty for me. Why have you set that pre-requisite? Would it not still be useful to you if it simply increased the efficiency of solving simple day-to-day tasks that you're not blocked on?

I'm still waiting for the concrete examples that back up the above commentary about how much of a gamechanger it is.

> I've essentially got 15-20 high-priced world-class consultants in every field that I chose to pull from, working at my beck and call

In other words, this comment from above needs backing up. I've seen such descriptions everywhere but no one has ever provided examples that corroborate this.

> > I've essentially got 15-20 high-priced world-class consultants in every field that I chose to pull from, working at my beck and call

I'll give you that: many of the claims are hyperbolic & ridiculous, including that particular one by the gp. The only way I can think of that statement being remotely true is if "high-priced" has no bearing on the quality of work of those consultants (though tbh that is often the case in reality).

Personally I have not found ChatGPT particularly good at anything I need to do. I have however found it very passable at many things I don't enjoy doing. It's reporting boilerplate is far more tailored & novel than anything I could ever put into a reusable reporting template. I dread writing reports: it's not a technical challenge for me but I gain no stimulus from it, so reducing the task to one where I just have to heavily edit something ChatGPT prepared for me is frankly incredible. The same goes for formal cold-call emails to people I don't know: another menial, unpleasant but necessary task.

It's also great at spitballing, and - as others have mentioned - quick shallow topic summaries of things you'd normally rely on a quick google & scan to get a quick topic summary (timely as Google gets worse and worse).

These are microgains throughout your day, nothing truly revolutionary, but that's the case for many successful tech we take for granted.

I wanted to use my GPX files to generate my own private hiking/running log.

With basically zero knowledge specific of GPX files, D3.js, or mapbox before starting I was able to quickly write a plugin that displays a path on a mapbox map as well as generates an elevation profile.

It would have taken me so long to do this before by googling and reading documentation with trial and error I would have stopped the project.

Now I have a working system.

For me, personally, it has replaced Google Search for the type of query that looks like initial research on a specific subject. "What is X?" then ask a couple follow up questions and I can quickly get a rough idea of some subject that I happen to need at the moment.

Then I go on HN and read these paeans from other technologists who say ChatGPT has completely changed how they work and is 10x better than using Google. I'd like to have that too! I just don't get it, it doesn't match my experience at all. And yes I did try the paid ChatGPT model for a month.

This is exactly my experience and what I am trying to get at. I start feeling that maybe I'm missing out, so I go give it a try. My direct experience shows that it spits out gibberish, but then people say things like it's a godsend but don't give any examples besides maybe it generated some HTTP API requests for them.

GPT+ data analysis gives GPT4 access to a jupyter back-end.

I was dealing with a noisy sensor in a factory production line.

I asked GPT to remind me which averaging functions might be usable to smooth out the noise, within the constraint of a PLC with limited memory.

I got GPT to simulate the output from the noisy sensor, apply each averaging function, and supply graphs of inputs and outputs. It was then pretty easy to eyeball which function was most suitable for the task.

I then gave GPT a code-style example and asked it to provide an (IEC 61131-3) Structured Text implementation using that code style. Which it did. This turned out to be pretty close to the final implementation (after careful reading and testing.)

Because it's so cheap to do (time and money wise), I used GPT to generate quite a lot of throw-away code to get to the final result. I probably wouldn't have considered this particular approach if doing it by hand.

This is because the client in question is in a regulated industry.

This week, we used it a bunch of times to help rephrase objectives while working in brainstorm sessions, allowing it absorb ridiculously large regulatory PDFs to give us summaries and ideas of what our greenfield project should focus on.

Things I’ve gotten value out of in the past week or two:

• Writing a job description

• Making a python script to automate some stuff in Asana

• Simplifying some management concepts so I could slack them to a coworker

All of these are things I could easily do myself. But with ChatGPT, they’re done 75% as well in 10% of the time and I don’t have to think hardly at all.

This year I used GPT-4 to write a significant amount of Terraform that was necessary to migrate an application onto AWS.

Writing Terraform, in my opinion, is problem that's broad but shallow. GPT-4 needed to do little beyond summarize documentation, but it was able to do so competently, and that was hugely valuable to me.

Conversely: In my free time, I've attempted to use it for a game-development side-project, and very little of its output has been useful at all.

I was also able to use it to generate software to check the blockchain to gather data I needed, again with no experience using the blockchain client libraries

Can you give some example prompts that will reproduce such an experience?

I could have looked up the individual parts and found the problems myself, but then I would have spent at least 15 minutes on it instead of 30 seconds.

1) I needed to display a very high resolution image to a user. I have experience in GIS and imagery, so I knew I should use an image pyramid in some way, and from previous experience assumed I needed some server to cleverly serve the tiles. But I didn't want to implement it myself, and googling 'map server' led to rabbit holes. I consulted with chatGPT, and while it gave me several fake solutions, eventually it suggested using gdal2tiles.py to create the pyramid and then serve it directly with a CDN. This never occurred to me, and is a much better fit to this problem. This saved a LOT of time (on either building my own server or fudging with other solutions) 2) I have a streamlit service, and needed to use some of my infra inside, infra that was written using async-await. Unfortunately, streamlit and asyncio don't play nice together (boo streamlit). I went to chatGPT hoping it would find me a way to make it work anyway, and after trying everything it suggested (which failed), I tried googling myself, and spent several days without a solution. Eventually, I went back to chatGPT and it suggested building a small HTTP service that would serve the results, and access it in streamlit using requests, no async-await required. It's a hacky solution, but significantly faster than reimplementing my streamlit dashboard in another framework or rewriting my infra without async-await. It saved loads of time.

I think you'd agree these aren't junior level issues. ChatGPT definitely didn't solve every problem I came to it with, probably not even most, and even when it did I had to intervene signficantly. I feel the more experienced you are as a developer, the less valuable it is. But when you need to tinker in a field you're not proficient at, or if you need to brainstorm a solution to a tricky problem, it can be a great tool. I understand why many swear by it. It takes a while to learn what sort of issues are good to turn to chatGPT with and which aren't, and also how to phrase those issues

I had searched all over the web for an example addressing my specific use case on google and couldn’t find one. GPT4 produced a working example for me and got me past that road block. I also use it regularly to suggest better coding patterns and I find it does a really good job at doing code reviews for obvious mistakes / anti-patterns

But my mind kind of works like: once I'm stuck I endlessly do my best to understand it, which sometimes mean that I grind to a halt. ChatGPT stops that entirely.

The fact that it always produces 2 to 3 bugs is fine.

So - I just reached out to Chat, and we started going back and forth, starting with "In linux, how many open sockets can there be at a time?". What's nice, about that wide open question, is you don't get a single answer, instead you get a briefing on Linux Sockets. File Descriptors, sure, but also memory, port range, TCP/IP stack limits, etc... It starts to lay out a roadmap towards solving the issue, answering the question you were interested in, rather than the one you asked.

I do a bit of back and forth on some scenarios, asking about /proc, and ss, etc.. seeing if I can track anything else out. And then, after spending about 5 minutes, and building context, I ask it "Is there anything else that can cause an error regarding too many sockets with low socket use" - at which point it lays out a number of scenarios, one of which is FD_SETSIZE.

So - we dig into FD_SETSIZE, and immediately that looks interesting - it's the limit that you can use with a select() call, and, even better, I get recommendations to use POLL or EPOLL (which anyone who has ever straced has seen a ton of).

I ask it how to determine FD_SETSIZE, discover it's 1024 on the client, which matches our low socket count, confirm that we should never increase the size of FD_SETSIZE, check the vendor code, see they've got it hard configured to talk with select() instead of poll - we recommend they give us a new build with poll() defined - and voila - problem goes away.

On to the next issue.

Where Chat excels is not in solving or completing anything, or, in fact, even being correct 100% of the time (it clearly isn't) - but it's an endlessly enthusiastic savant intern - frequently wrong in hilarious ways, but always willing to dig in and pull on technical threads and come up with things that I can try with a bunch of rapid iterations, and close off an issue. It's willingness to write code that is 90% correct just reduces the time, and cognitive load of constantly having to do it all yourself.

I’ve had to sharpen my skills at verification and cross-checking, but the volume of pretty good insight from a much wider domain than I myself could consider has been so worth it.

It’s also worth noting to that I feel like I used to get this type of help, roughly, from good search engines. Maybe not quite as spoon fed but pretty good. But now… just blogspam and paid listings.

"I've essentially got 15-20 high-priced world-class consultants in every field that I chose to pull from"

And then here said:

"It's an endlessly enthusiastic savant intern - frequently wrong in hilarious ways"

And those feel very different to me.

The second one feels more similar to my experience This is what I spot with ChatGPT (and advocates of it): in the abstract, it is a genius, but whenever we look at a specific example, it's not like that at all.

The main argument is that you can give it a block of real-world data like an email or code and take advantage of collective knowledge to identify outliers like bugs, bad grammar or incoherent writing which translates exactly to code semantics too.

I can also feed it a list of web accessibility issues and have it sort the list into most/least critical, along with providing references to the specific WCAG criteria. It has occasionally stumbled on this task, but again massive timesavings.

I also use ChatGPT for filler text in designs. Yes I have to build on what it writes, but it's way better and quicker than what I'd do myself. I know our communications team is doing this x10 more than I am.

I use ChatGPT 4, GPT-4, and Copilot every day. It is an "average intern" at many many things. Here's how I feel it helps me:

* Its interactivity lets me learn a lot (superficially) about new topics. I can then expand that with research of my own

* It helps me think outside the box or from other perspectives when thinking about e-mails, proposals, real-world scenarios

* When exploring a new language, framework or technology, it points me in the right direction.

* For quick scripts, using the code generation/analysis feature, if I direct it right (i.e. layout "the plan" beforehand and ask it to work the rest on its own), it gets a lot of it right pretty fast, saving me some time writing the code, figuring out the right libraries and the nitty-gritty details.

* It is great at giving ideas for why something might not be working.

Real things I've done with it:

* Discuss ongoing negotiations with clients, trying to better my proposal and better understand the clients point of view.

* Learn more about managerial or "business-y" topics, by allowing me to discuss things with it and iterate on that with my own research. It is a valuable "white board" to discuss with.

* Adjust my e-mails so they are more appropriate to the situation. This can involve changing the tone, shortening them, adding more detail, etc.

* In general, i've used it to find flaws in reasoning when dealing with people. For example, it has helped me question my own client proposals or approaches, by specifying where I was lacking (e.g. because I was vague, or pessimistic, didn't give a mensurable objective, seemed to ignore a client's request, etc)

* I use a command-line utility from the shell which lets me ask it to do something and then have it do it. I now use this with some frequency to just write the commands I would have to google because I haven't memorized. Things like ffmpeg or imagemagick arguments. Or combinations of grep, sed, ls, find, git, etc. Here are some examples:

i) "merge jpgs in this folder in order onto a pdf. each jpg should fill the page 100% without margin. Avoid complex calculations".

ii) "zip this folder with maximum compression and password 12345678'".

iii) "git remove branches matching pattern sprint-* remotely'"

iv) "use imagemagick to convert to jpg lossless and rotate 90 deg clockwise ~/home-en.png"

v) "Add _en before .jpg in all files in current directory. So they become _en.jpg"

vi) The list goes on and on...

* It has helped me cleanup nginx config files

* I have thrown code at it which I suspect has a bug. With a bit of context and some back and forth, it has helped me find it.

* In languages or frameworks I don't use often, it really shines. It has helped me write several applescript scripts which I have also cobbled together to create Alfred workflows. If I need to code something in a language I don't often use, what it produces is good enough for me to iterate on.

* It has helped people at our company improve their copywriting (when used with a lot of care)

* I have used it to help me critique my own poetry and improve how I approach poetry in general. Highly subjective, I know

* When trying to figure out how to use apps I don't often use, or dealing with unexpected behaviour in them, it often helps me find the issue. Notable apps in this category include Photoshop and Excel.

* I don't often do frontend so I'm particularly bad at styling and organizing things. When I ocasionally have to do frontend, it often gives me the initial "skeleton" very well.

I have seen many people try to use these tools, and here's where I think they SHOULD NOT use them:

* For facts, obviously -- which is unfortunately what many people actually try to use it for

* For writing (without check) most of your e-mails our posts, especially with very little context

* For ideas where you need "just enough creativity". It's a very fine line. Think brainstorming ideas for UI elements in a new website according to specific brand guidelines

* For incredibly specific details or overly complex tasks. You can't often have it write your 200 line function out of thin-air!

It is clear that GPT provides me with:

1. A better search engine, which skips past loads of outdated content or SEO-laden bullshit. It's just a shame that I can only use it this way for more specific "creative" or "problem-solving questions" questions. For important fact-based info, I always have to check what it says.

2. A partner with whom to discuss ideas and iterate over them, especially on topics I don't know. If anything, it's a great rubber-duck

3. A way to forget the underpinning of some of what I do and approach it with natural language. I find myself asking it to write my bash sequences instead of thinking them up myself. Of course I check them and 99.9% understand them, but it's just so much easier to have it do it for me.

4. (Copilot is an upgrade of my IDE. And a great one!)

If modern Google (and Bing and everyone else?) weren't so shit, I wouldn't need 1. To this day I still first go to google, eventually give up because it doesn't answer me properly and then go to GPT. It's ridiculous. I think 1. brings a lot of value to many many people. Google is so absolutely shit nowadays I can't believe it -- and the more I use GPT to get a comparison baseline, the more I am shocked. 2., 3. and 4. are incredible additions to my workflow.

I truly believe we are heading in the direction of interacting with computers via a new interface. Even writing code might eventually be affected. Perhaps, like we have built high-level languages on top lower-level languages and machine code, abstraction over abstraction, we might end up with some form of language writing which uses (very well controlled) LLM technology under the hood. Instead of directly writing the code, we tell it to write it. We still need to be smart to pick the right algorithms and structures, but no longer have to worry so much about writing the nitty-gritty syntax or details. Maybe?

Using GPT is its own art. Crafting the right prompts and getting a feel for how it works is very important. It is also essential to have critical-thinking and know when to question what it says and when not to.

A friend of mine told me this: "The companies who strive on hiring loads of shit people to do work are the ones who will suffer more. The others, which have hired smart people capable of critical-thinking, will benefit. GPT obliviates the work of many small shit people. Pair GPT with someone who is already smart and has very good problem-solving or critical-thinking skills, and now you've got a team that can obliterate the teams of many small resources.

I don't know if OpenAI will succeed or not, but I do know this technology is absolutely life-changing for many, and going back to a world without it being this acessible will likely be a net-negative.

But ChatBLT was ‘trained’ using SEO-maddened bullshit thus your getting the statistical mean of all that bullshit.

The first rule of the internet is don’t believe everything on the internet. Hitching your financial wagon to it seems crazy to me.

How? Maybe because it was trained on lots of data and therefore isn’t just “spitting the first thing”, or perhaps it’s because it’s actually just answering the question I asked, without muddling it in a page with 50 other similar questions designed to up the SEO ranking. Whatever the reason is, it clearly happens often.

I’m not advocating for not fact checking ChatGPT. Quite the opposite. But even with that caveat, it adds tremendous value, at least to me.

I suspect it is because I tend to have off piste questions which are not recipes or high-level ideas. It is where it'd be most useful to me, but it is at the same time where least training data is going to be.

It is also where actual legitimate experts are most useful: the special sauce, not the meat and potatoes.

Would have taken me much more time to figure it out than what chatgpt did at first try.

TL;DR don't ask it anything to do with logic, but anything to do with documentation and stuff like that it's pretty good

I mean it. I'd pay money to watch streams of someone using chatgpt to solve non-trivial problems.

For example, trying to get chat gpt to do something very simple for my work, like implementing a convolution it took me in circles and circles.

It gets the general idea right, sure. But it actually makes significant minor errors that ended up being more confusing than helpful.

I pasted it to GPT4, said "I think this code has a concurrency issue, can you find it for me?" and sure enough, it told me what line of code was causing the issue.

It is also great for doing things that I don't do often. For example, I almost never write bash scripts (like once or twice a year). Now days I just use gpt4 to write the script for me. For example a couple days ago I needed to walk through a folder full of files and pass them one by one into another program, and if that call was successful, append the processed file to a list of files that had been successfully processed. Trivial, but again, I never write bash scripts. GPT4 had it up and running in a few seconds.

I can throw a file at GPT4 and ask it to create mocks for my objects so I can go ahead and start writing unit tests.

Awhile back I had setup a new express server, but after I added my CORS config everything stopped working. I spent an hour debugging, no luck. I asked GPT4 what the issue was, and it noticed I had defined CORS on the wrong line (after I had started my server up). Oops.

GPT4 is also great at coming up with sample command line options for utilities. Sometimes good examples in man pages exist, sometimes they don't.

Honestly, I don't use GPT4 as much as I should!

In my example I said "I'm trying to translate my React Native app into multiple languages. I need an object that looks like this: <example JS code>. Can you generate this object for French, German, <insert more languages>, and for the following words: <words I needed>."

Luckily I know basics in a few languages and the words were simple enough that I could easily verify it wasn't a hallucination. I also know enough JavaScript to know that it was syntactically sound, but my IDE would've told me if it wasn't.

I admittedly give it a vague question "where the signals overlap"

In context of the original thing I was trying to do, I was filtering a receive signal and trying to remove the delay from the convolution

I remember the first time I watched a Netflix developer livestreaming their workday using an impressive neovim setup. It was eye opening.

I need the same experience for this ChatGPT fanaticism.

the goal here is to accelerate productivity, is it not?

It’s written at least 20 python scripts with me and almost all of them have been close to perfect almost on first draft.

Examples I can think of where it gave wrong info was:

- I asked to compare the size of the MacBook m1 and the 16" framework laptops

- I asked how to add the nixpgs-unstable channel in NixOS.

- I asked to help write a power shell script to create AD accounts.

It's like if an alien visited and the first place they went was to a convention for Special Needs people and then they decided that humans were too stupid to talk to. Or rather, there was a Mensa convention next door with a $20 entrance fee and they figured it wasn't worth it.

Or rather, for $20 everyone at the Mensa convention agreed to log in to a special app and wait 24/7 for any time you have a question. Then they would instantly select the most knowledgeable among them in that subject and immediately enter a reply incredibly quickly.

The first one is open ended and I don't think M1 came before the "cut date of knowledge" in chatgtp. The 2nd one is too niche, likely not enough data to grab.

For the 3rd one, I don't know if the "date of knowledge cut" isn't recent enough - powershell became a thingy more recently and isn't wide adopted.. but I would expect to be a good fit

Claude Instant will not give you the highest quality answers.

Two interesting things it did for me lately: insisted that 8 + 6 = 11 and also proved that P = NP. I don't know which solutions it may be providing you, but it can't be anything too complex, or at least not too abstract.

edit: typo

I think the "GPT is amazing" vs "GPT is useless" debate is just going to get more confusing as more versions are released.

LLMs are not good at search, math, encyclopedias, logic engines. Maybe some day they will be, but not yet.

For example, "I'm setting up a new TypeScript + Svelte app, I made some changes to the config that I thought were good, but when I try to run the dev server, I get this error." And then paste in your incomprehensible ten line error.

Any time you are using some software tool that you're not very familiar with, and you get an error message that you don't understand, try asking GPT4 to explain it to you.

It isn't so great at deep, theoretical, algorithmic questions. "NP reductions" are probably not a great fit.

Think of it as a research assistant that has a broad understanding of every technology in the world, but isn't as smart as you are about your specific area of expertise.

It can be very helpful to guide you in a direction maybe you didn't consider looking into to begin with, but I don't fully trust it.

There was a fun example I had a few months ago where I wanted to see how well it would do with being asked to solve a problem iteratively instead of recursively, something like: "generate all valid, distinct permutations of a provided string given that you have a dictionary to check valid words against" and it got most of the problem correct, but when asked to fix anything it would go right back to a recursive solution with the same issue appearing, or in some cases a new issue.

It got me most of the way there with some edge cases I needed to handle myself, but it definitely seemed like that was as far as it was going to be able to go

The real issue will be that majority of users will take the answer face value without even knowing if that answer is a good one or just nonsense.

"The AI said..." will create lots of issues. Time will tell.

So for an MBA major, 80% accuracy at 5% of the cost may be amazing but for me as an engineer and a person who cares, the inaccuracies are catastrophic.

I’m open to suggestions on how to work with this.

So, just like an expert consultant?

Seconded.

Let me tell you what I used to do.

First, Imagine I have an error executing code or an error running some bit of 3rd party software. I go to Stack Overflow and search. I find posts related to my problem and I spend a great deal of time trying to shoehorn existing answers to my specific issue. Sometimes, my shoehorning works and I fix my problem. Other times, it doesn't work, and then I post on Stack Overflow myself. And I wait ... and wait ... for a response. Sometimes I get a response.

Now, when I have this type of problem, I tell ChatGPT, "Hey, I'm trying to do <xyz> and I'm getting this error <abc>. Help me troubleshoot this. And it almost always helps me fix my problem. And it's ~10× faster than Stack Overflow.

=============

Second, there are times where I have write code to do some relatively 'complex' data manipulation--nothing sophisticated, mind you, but stuff like, "I need these data columns rearranged based on complicated logic. And I need the text in columns A, X, AQ, and F merged, but only if <blah blah is true>. Otherwise, Just merge text in Columns A and AQ, except if the date in column ZZ is after January 1, 2019.". I can do this stuff on my own, but: a] it's cognitively draining, b] it takes time, c] I often make silly errors due to the complexity.

ChatGPT is, again, an order of magnitude faster than I am. And it makes _fewer_ errors.

It still makes errors. And I still have to know what to look for to catch those errors, but it decreases my cognitive load tremendously.

edit: I haven't used Stack Overflow in 6 months. And "Googling" is Plan B.

=============

Edit 2: I recently had to write a sympathy letter to someone whose husband died.

I knew the general ideas behind what I wanted to say, but I knew I wasn't going to write anything great.

I fired up ChatGPT and said,

"write a sympathy letter to <x>. Tell her that I didn't know her husband well, but the few times I met him, I could tell that he cared deeply about you<x> and his daughter. I know his daughter well and I think she gets a lot of her great qualities from him and you. Tell her I don't know what to say in times like this. Keep it short-ish. Avoid schmaltz and sentimentality because <x> isn't that kind of person."

It gave me about as perfect a letter as I could have asked for.

I suspect this is one of the differences between people that find GPT helpful and people that don't--it seems much more potentially helpful when you are in a less familiar environment. In my work, when I run into unexpected behavior, if I knew what question to ask then I would probably already know the answer too. Until the day I can feed in an entire codebase and ask "What's wrong with this? vaguely gestures at everything", I don't expect that to change significantly.

You just found your path to become a better developer than the majority then

OTOH, I have found a use for stablediffusion that actually resulted to some income.

That would be amazing if Google, Microsoft, Amazon, or Meta sat on their hands while OpenAI got that big.

Google Used to be able to handle a lot of the non-procedural questions - but somewhere in circa 2022/2023 something started happening to it's results and I started getting back mostly SEO churn, to the point I was going back to using manuals and having to dig in and learn the fundamentals on a lot of things - which is unsustainable if you are touching 30-50 different technology stacks.

Chat changed all that - I can now go 3-5 levels deep in some stack, ask some incredibly nuanced question, get a reasonable answer that points me in the right direction, close off the issue and then move onto the next one.

IMO, this is the crux of the problem, and it’s caused by allowing dev teams to use whatever stack they want, without also requiring them to support it. More cynically, it will always be the case if teams are allowed to use whatever they want, regardless of “service ownership” – because when there’s a SEV-1 and the owning teams have no idea what’s wrong, the SREs are going to be told to fix it.

Being given some code and reviewing it is a lot quicker than writing that code. Copilot is great. Half the time it spits out the wrong answer but you can see what it was “thinking”.

Phind giving you an answer and references lets you quickly double check. Sometimes it hallucinates but the answer and references combo is much better than a Google search which in turn is better than nothing.

Of course I recommend people not use AI for everything. I will go straight to MDN for any WebAPI question and use —help as my first port of call on the command line. This is like your L2 cache as a developer. Using AI for everything is like swapping to disk for everything.

Sometimes you need to go back and forward a bit, I tell ChatGPT it’s wrong and the error message and then it spits out the correct result, sometimes I need an algorithm tweaked because it has assumed a wrong constraint on the problem, again just explain it clearly and unambiguously and it will make corrections, there’s only been maybe 1/40 problems I couldn’t get a correct answer on after (sometimes a lot!) of back and forwards.

I am not looking for a perfect oracle, I am looking for something to write 80% of the code and then I’ll fix it up. It’s still way faster this way, especially in domains I don’t know, E.g. I just learned CUDA with ChatGPTs help.

It’s not perfect, and neither am I, but it doesn’t have to be perfect to be useful, you can get to millions in revenue through 80% solutions

Learn the limits of your tools.

The value and time it saves makes sense for folks who struggle with a search engine (many) or doing tasks that are typically considered menial, like writing emails or coding boilerplate.

However, if you can grok Google and don't mind doing tasks like that (I personally don't mind coding boilerplate stuff, especially since I can learn how the framework works that way), ChatGPT's value is limited (at least in my experience).

Example: I was struggling with a Terraform issue the other day. I used ChatGPT 4 to help me work out the problem, but the answer it gave was really generic, like mashing a few of the top answers on SO together. It also didn't answer what I needed help with. I knew enough about how Terraform worked to Google for the answer, which I eventually did a few minutes later. I could have kept crystalling my question for ChatGPT until I got what I wanted, but Google was easier.

I'm also not a huge fan of us just being okay with trusting a single and extremely corporate entity (OpenAI) being the de facto arbiter of the truth. At least Google shows you other options when you search for stuff.

^ Isn't that what folks used to say about programming in assembler? How much time do I want to spend learning frameworks (beyond what I already know) vs. how productive do I want to be?

You don't need to "struggle" with Google to get value out of it, you simply need to value your time.

If you want an answer to a question, why waste time reading through pages of search results when you can have an AI do that for you, reporting what it finds?

No, it's not perfect but it's pretty damn useful.

LLMs aren’t AGI. They’re far it. But they have massive uses for reasoning on available context.

I’ll give you an example. I’m trying to set up some bulk monitoring for api across 200k jvm s. The api documents are horribly out of date. But I get the raw uri on the monitoring tools.

I can just get these uri, send them into chatgpt and ask for a swagger spec - along with a regular expression to match the uri to the swagger api. It figures out the path and query params from absolute paths.

Sure I could try to figure out how to do this programmatically using some graph or tree based algorithm. But chatgpt basically made it possible with a dump Python script.

Of course I may still need a person to fill in these. But just getting a swagger spec done for a thousands of services in an afternoon was awesome.

This type of rhetoric is part of the reason so many compare the current crop of AI to cryptocurrency hype: proponents constantly telling others to shove the technical solution into everything, even where it’s not necessary or worse than the alternative.

I know where you're going. I've had folks say to me: "I really like co-pilot because it enables a beginner like me to write code". This sentiment often comes from folks having non-technical roles who want to create their own software solutions and not have to deal with engineers. I roll my eyes at that one.

You need to be able to spot specific areas of acceleration. Not just tackle it as a hammer for every problem.

I could also split the uri by service names. That helped parallelize my questions. It wasn’t just dump the data in. There was some cleanup behind the scenes that I had to do.

They're completely different products.

But in terms of value creation they have turned numerous industries and jobs on their head. Things like copywriting or how they are destroying stackoverflow and quora. The next lowest fruit they are disrupting is front line chat/email support - this is usually never part of a core product but the market is massive, almost every company needs support - look at Zendesk or imagine the costs of Uber's offshore support army.

They are going for the AWS platform approach - every niche GPT wrapper that gets a modicum of success and has happy paying users, these users likely are likely to stick because it improves their work in obvious ways. OpenAI gets their slice, think of how AWS made it easy for anyone to spin up a service, sure some failed but the hurdle is much lower - with such a powerful general model, I don't need to spend millions training my own to launch. The issue for them is it'll likely be less sticky if competitors/open source models catch up - hasn't happened yet but it might.

I've never seen such disruption to ways of working in so many industries in my lifetime. If you're on HN you may not see any use in your work if specialised, but at the entry level (majority of workers), doing writing work in 30% of the time has been game changing.

But since software scales so ridiculously well, that their cloud offering still manages to beat it on profit.

OpenAI had just landed a new class of subscription that scales like AWS, has B2B hooks like AWS, can be the engine behind entire classes of future unicorns like AWS... but then also has widespread consumer value and brand recognition like Prime.

By that measure it's hard not to be bullish.

Before ChatGPT, to find the answer to things like "how do I set up Gunicorn to run as a daemon that restarts when it fails" I would have to endure hours of googling, snarky stack-overflow comments that I shouldn't do that, etc., but as a solopreneur without access to a more senior engineer to ask, it's been fantastic. I've been quite skeptical of machine learning/AI claims but I feel like I'm experiencing a genuine case of a technology that's proving to be so much more useful than I had imagined.

Do you mean, “have Gunicorn keep N workers running?” If so, that’s in the manual (timeouts to kill silent workers, which defaults to 30 seconds).

Or do you mean “have Gunicorn itself be monitored for health, and restarted as necessary?” There are many ways to do that – systemctl, orchestration platforms like K8s, bespoke scripts – and all of them have tricky failure mechanisms that a casual copy/paste will not prepare you for.

Blindly using answers from ChatGPT is no different than a random SO post, and you are no more prepared for failure when the abstractions leak.

Getting straight answers will be detrimental in the long term, I fear. It feels like living in a box, and watching the world on a screen and the person answering my questions is mixing lies and truths.

> snarky stack-overflow comments that I shouldn't do that,

and realized they're probably using the chatgpt equivalent: nice, corporate answers that you probably shouldn't use

Also, would you really Trust AI for everything ? I wouldn't. Nothing beats human element. At best, AI should be used as a supplement to speed things up which it is great for. I personally would never rely on AI to do everything for me. Not to mention that it cannot be 100% correct.

> I just don't see this company (or any others for that matter) GPTing their way to AGI

> I'm not saying the company will go bankrupt but I'm also not buying into the hype that it's going to become the next Google or better / create AGI for us all.

What I'm missing is the connection between AGI & profitability. OpenAI has huge revenues from ChatGPT which look set to continue - they're distinct from cryptocurrency in that those invested in them are so on a service provision basis rather than a speculation basis.

I'm thoroughly unconvinced we'll ever see AGI - I see zero connection between that and OpenAI being successful.

I am bullish on AI but don't see AGI happening, yet I developed AI solutions that solved real world problems, and made companies tons of money and helped people solve non-AI, non-IT problems.

So, I never buy the AI-crypto-equivalence.

But so far to me this seems to be almost universally loved by programmers while I don’t really know anyone else who uses it at all.

I think after the past 15 years which saw some of the most rapid technological advances in history along with the greatest bull market in history, people’s credulity is off the charts.

But to me something just feels off about this entire AI/NPL complex. For one, I agree that it’s largely oversold on features. Also, every single enterprise software company is attempting to jump on the bandwagon and everyone on LinkedIn seems to be posting about it every day. Most people who talk about how revolutionary it will be have absolutely no track record of being correct on similar calls and on the whole if I had to bet, probably were highly skeptical of new tech over the past decade that was revolutionary.

I also agree that it feels very similar to crypto. I don’t think it’s a coincidence that both were largely enabled by advances in nvda chips. It may sound absurd to most but I actually believe nvda is the most overpriced stock in the market now by a large margin and is sort of in its own bubble. There has been a head long rush to stockpile their chips in anticipation of npls taking over but I predict it is going to result in an eventual glut of oversupply that’s going to put downward pressure on the semiconductor market potentially for a year or more.

If AI-generated code is considered acceptable in your project then you aren't using a powerful-enough programming language. And you're paying the cost in code bloat.

How many Coq programmers find ChatGPT useful? How many nontrivial Coq programs written (and not merely memorized) by ChatGPT even pass type checking?

If you're considering AI-written code then you have a bloaty-code problem. Letting an AI write bloaty code conceals the symptoms (keystroke count) but tech debt will still kill your project sooner or later.

That said, I do agree with the general notion. I find the more verbose the language, the better the help. Dense, more expressive languages fare worse. I’m referring to Python and Rust in my case, so one factor is of course massively larger training corpus for Python, and relatively more churn in Rust.

That’s a good example of a subpar use for an LLM. Dictionaries have existed since before computers. Digital dictionaries are faster, more reliable, and less power consuming than any LLM.

Parent's wife is presumably a smart person who found a way to use technology in a way that benefits her, without knowing a lot more context, I think it's a bit silly to leap to the conclusion that she is "doing it wrong".

It's a chat, she is talking about a subject, and then she can expand on a word, all very naturally, as she is talking with a human.

That's very different than having some subject you're reading about and then having to carry a dictionary which has no context about the subject, and doesn't talk back to you

Perhaps you should consider I have done that, instead of assuming things without knowledge of my experience and only worrying about repeatedly telling other people they are wrong about a subjective experience.

As you keep ignoring important points, including reliability which is paramount for the subject at hand, unfortunately I can no longer assume the conversation is happening in good faith.

OpenAI itself will be fine though. Their lead has a snowball effect with all the training data they get. And I'd guess they will succeed at their regulatory capture attempt, and create some horrendous pseudo monopoly. Meanwhile, they can just implement what the most successful wrappers do themselves.

When I've seen colleagues visibly use it (i.e. mentioned in commit messages) that confidence has rubbed off on them.

Given that, why would I believe it when asking something medical, legal, historical, or otherwise outside of my domain or that I can't somehow verify?

LLMs are far from the only thing you rely on that will confidently lie to you.

> Every time a new study comes around saying “actually, everyone should be drinking 3 glasses of wine a night”, do you take a trip to the liquor store?

No, and doing so would be an example of blindly trusting something that you haven't or can't verify, so that supports my argument?

> LLMs are far from the only thing you rely on that will confidently lie to you.

Sure, but it's a new class of thing that will and does and yet people are trusting or haven't yet learnt that they can't. I mentioned seeing people trust it via commit messages; I don't see SO the same way, people generally realise they need to verify it, or it at least has a voting mechanism as a proxy. With GPT so far there seems to be a lot more assuming it's correct going on.

But with ChatGPT we're not saying disregard what it says, we're saying only disregard some of what it says and don't disregard what it says in future. Which becomes a lot of work to check everything it says every time.

Crypto, after more than a decade, has been useful only to criminals and scammers.

Of course, gpt-5/6/7 can become more valuable to end-users, but that's the second reason I'm bearish. LLMs are powered by exponential growth, and no exponential growth is infinite. We are already using a significant part of all existing data, and going up more than 1-2 orders of magnitude in either data or compute feels unlikely unless there is some breakthrough. Breakthroughs are hard to predict, but they are generally rare, and likely there won't be one soon.

I feel that some people lack creativity to use it.

GPT is as good as the user is at posing good and well defined questions and tasks.

Its ability to perform few shot learning is astounding.

I suspect I'm fairly alone on this. They'll probably do well without me.

Most people that even know about it probably don't mind. I can't even verbalize why I do

I've had a play around with some OpenAI-powered sites and it is neat how much it is capable of, but I feel uncomfortable typing personalized prompts or detailed questions into a system where I know everything I type is going to be harvested. You could argue that by commenting on HN or posting anywhere in the internet everything I type is also going to be harvested (perhaps into the very same models), but that contract was always clear. There is a difference between companies using information I have chosen to share publicly, and companies doing the same with what is presented as a private exchange.

But once they can fit that mini GPT into my pocket, and the learning it's doing is truly personalized to my own install... for me that will be a much more appealing product. I guess the technology will get there, eventually.

You could do it now. Apple computers with a lot of RAM are pretty good at running Llama2.

My workstation has allowed me to dabble - I'm familiar, a unified pool of memory does very little for me.

The experience with self-hosted stuff leaves a bit to be desired, both in generation speed and content.

The software needs work, I'm not saying we won't get there... just that we haven't, yet.

With a ridiculously beefy system I can eek out some slow nonsense from the machine. It's neat, and I can do it, I just don't find it very useful

I’m guessing you haven’t actually been using it personally beyond some superficial examples.

Once you use it regularly to solve real world technical problems it’s pretty huge deal and the only people so far that I’ve met who voice ideas similar to yours, just simply haven’t used it beyond asking it questions which it isn’t designed for.

When anything gets more complex than that, I feel like the main value it provides is to see what direction it was trying to approach the problem from, seeing if that makes sense to you, and then asking it more about why it decided to do something.

This is definitely useful, but only if you know enough to keep it in check while you work on something, or worse if you think you know something more than you actually do, you can tell ChatGPT it's wrong and it will happily agree with you (even though it was correct in that case). I've tested both cases: correcting it when it was really wrong, and correcting it confidently when it was actually right. Both times it agreed that it was wrong and regenerated the answer it gave me.

> I'm using python and my string may contain file paths inside, for example: (...) , For anything that looks like a filepath inside the string we should replace its full path, for example (..)

> Can you write me a python script to kill processes in Windows that no longer belong to a process named "gitlab"

> I want to write a test for it that mocks both sentry_sdk and LoggingIntegration so that I can give my own mocks during test

> I want to create a Python script

It should be able to be called from the command like, like below (example)

Write me the script

;;;

All real examples from last week that took me 1 minute to be solved instead of of googling or creating from scratch / thinking about it

When will humans live in space?

Why am I depressed?

When will world war III happen?

Compute this math equation (function calling and compute engines will help with this)

The technical people who reject it are quite curious psychologically, my personal suspicion is they are threatened by it. They get hung up on small hallucinations and then almost get giddy when it produces something “wrong” in some way. I don’t understand why they fail to understand it’s crazy importance, I mean it’s read everything and without an agenda other that the material it was feed, no twisted incentives. They things I’ve had it do with me are mind blowing, my guess is the people who understand it and how to leverage it will increase their own productivity so much that it will reshape economies and put many people out of work that don’t learn how to use it.

Definitely the most revolutionary development of my life, as I approach 50 and have been coding since 12, and believe it or not but working professionally coding since I was 19. Internet and iphone have nothing on this development with LLMs.

This is the kind of hyper sensationalism that I'm talking about. Do you really believe that, or is this you extrapolating to what could be possible in the future if the technology keeps improving? I feel like that is where a lot of the arguments always ended up with crypto advocates as well, if you had doubts or questions about how big of paradigm shift this was going to be for the world, you just didn't get it yet because you couldn't connect the dots this early on.

I'm not doubting that the tool is useful, or that ChatGPT is quite an accomplishment, but I just don't see it "reshaping economies" anytime soon.

I don’t just believe it I already see it happening directly. I have made changes in hiring strategies and employment situations based on massive increases in productivity from using LLMs.

The difference in productivity of developers that embrace the new tools vs those who don’t is very obvious in my opinion. It’s probably the next 18-24 months when the impact becomes more obvious on larger scales.

The underlying tech is amazing. Where LLMs are headed is wild.

I have just lost a lot of confidence that OpenAI will be the ones getting us there.

The chat niche was an instance of low hanging fruit for LLM applications.

But to design the core product offering around that was a mistake.

Chat-instruct fine tuning is fine to offer as an additional option, but to make it the core product was shortsighted and is going to hold back a lot of potential other applications, particularly as others have followed in OpenAI's footsteps.

There's also the issue of centrally grounding "you are a large language model" in the system messaging for the model.

So instead of being able to instruct a model "you are an award winning copywriter" it gets instructed as "you are a large language model whose user has instructed you to act as an award winning copywriter."

Think about the training data for the foundational model - what percent of that was reflecting what a LLM would output? So there's this artificial context constraint that ends up leaving to a significant reduction in variability across multiple prompts between their ChatCompletion and (depreciated) TextCompletion APIs.

They seem like a company that was adequately set up to deliver great strides with advancing the machine learning side of things, but then as soon as they had a product that exceeded their expectations, they really haven't known what to do with it.

So we have a runaway success while there's still a slight moat against other competition and they have a low hanging fruit product.

But I'm extremely skeptical given what I've seen in the past 12 months that they are going to still be leading the pack in 3 years. They may, like many other companies that were early on in advancing upcoming trends, end up victims of their own success by optimizing around today and not properly continuing to build for tomorrow.

If you offered me their stock at a current valuation with the stipulation I wouldn't be able to sell for 5 years, I wouldn't touch it with a 10 meter stick.

To me, this is the most important part of ChatGPT. GPT-4 has some massive shortcomings, but to me it's clear that this road we have started to head down is producing actual intelligence, in the real sense. 5 years ago, AGI felt completely intractable to me. Now, it feels like an implementation detail.

[1] https://twitter.com/tegmark/status/1709572469978231063

The problem is, this sentiment of path towards AGI assumes that everything (i.e our physical reality) can be compressed. Which is highly likely not true.

The future of AI is basically better and better compression with more data sources. You will be able to do things like ask how to build a flying car in your garage, and it will give you step by step instructions on what things you need to order from where, including CAD drawings to get custom made CNC parts, and how to put it all together (including software to run it).

As far as AGI goes, its possible that through AI assisted technology we will be able to measure synapses of a human brain in fine enough detail, or somehow mass surveil the entire population of earth and derive human brain models from that. And then, with optimizations on that, we can potentially arrive at the most efficient human brain in terms of some metric, and we will have very good robotic assistants, but nothing really beyond that.

Would my life become worse without crypto? Actually it became better - I sold all my crypto, coinbase made it painful to deal with them and they jacked up transaction fees. That money is now in good ol’ stocks.

So OpenAI specifically, I can’t say but AI in general that is trained on the entire knowledge set of humanity and that can reason from it - that will become ever more valuable.

I'm a teacher who is constantly learning new things. I can learn things I would have never been able to learn before because of AIs like ChatGPT. My students are learning more and faster than ever before.

Learning Management Systems like Canvas and Blackboard made a lot of money. I could argue they are obsolete now.

No, it objectively has not. Maybe it has changed how you teach, but “education” is much larger than any of us. Until school curriculums around the world incorporate ChatGPT—quite the dystopian scenario—they have not changed education.

I'm spending most of today updating my Canvas courses, and Canvas is obsolete. Students are much better off asking AIs how to do thing and what they should learn next rather than working through my Canvas courses.

2. School are slow to adopt new techs

I use Bitcoin regularly, because I live in a third world country where it's really hard not to get your salary seized.

I use ChatGPT every day for lots of things and it has replaced Google search for me. And StackOverflow, of course.

Notice how I said BITCOIN and CHATGPT. Not "crypto" and "ai".

ChatGPT made one great product. Bitcoin is one great “product.”

But the successors in the same category aren’t doing anything wildly more useful than the original “killer app.”

Crypto never really revolutionized finance, it just provided one solid digital currency product. Smart contracts and NFTs went pretty much nowhere and I struggle to identify a way that they are used in a widespread manner.

You and I are using ChatGPT regularly and it helps us quite a lot, but it hasn’t revolutionized life nor has it turned me into a 10x developer or something like that. It’s a service that is collecting $20 a month and that’s about the extent of its economic value so far.

In other words, “Replacing Google and Stack Overflow” is arguably not that exciting.

(Then I end up going back to SO/Google when ChatGPT tells me shit that is wrong)

I do think LLMs have way more potential than “crypto” but it remains to be seen how much more that is

All the crypto killer apps lost the plot, especially NFTs.

As for LLMs and more specifically, ChatGPT, they are not a niche thing so I agree in that their potential is way bigger. I'm not yet sure what that is, but I think it will change things profoundly. Replacing Google/SO is just a side effect of something bigger. But that's just my humble opinion.

Imagine if they kept doing that despite having your credit card information because you paid for Pro, which more or less proves you're an adult who deserves the presumption of innocence/good-actordom.

Lastly, imagine that you do this for all users despite the fact that it is known to reduce the intelligence of your output.

(I'm bullish on self-run models)

* Problem statement *

The actual value of GPT is spontaneous creation of spoofed data. Some of that output answers really tough questions. Stop thinking at this point and reflect.

* Value assessment *

There is some amazing potential there for performing test automation where either large data samples are demanded or the sophistication of output exceeds exceeds capabilities of prior conventions. Taken one step further there is further value in using this authentic looking out for testing against humans for market acceptance testing or bias validation.

* Real world use *

When real world use significantly departs from the value assessment there is a problem. Faking student papers or hoping the technology writes a legal brief for you forces a complete realignment of potential revenue model with different clients paying different amounts.

* Expectation *

Unless this technology can actually make money in a reoccurring and sustainable way past initial trends it will be an investment black hole just like crypto.

My company also got it immediately and is rolling it out globally.

GitHub copilot is already helpful. GitHub copilot for docs (announced at GitHub next) is a game changer.

I used openai to reformulate emails and suddenly got positive feedback about my announcement emails.

I communicate with openai German and English how ever I see fit.

It's very hard NOT to see it than the other way around.

And we got so far with only one company pushing this!

There is no option for the others to alsolo out tons of money into ai.

And besides openai, ai/ml is huge in what Nvidia and others are doing with it. Texture compression, 2d and 3d generation.

What we see is also potentially something like a new operating system.

And it makes it so much more accessible.

I never had a tool like llms which are able to take a bad copy pasta of a pdf and pulling out facts from it.

I did not foresee that you could mix input languages. That’s fascinating. Multilingual people use languages for different purposes, often in the same sentence. Ie: technical jargon in English, a joke in Arabic, etc.

I expect the difference in connotation/feeling/mood to be less relevant for an LLM, if you’re working with facts. But there was a recent post showing LLMs performing better when you said you were stressed/scared. Did you notice any such differences for your multi-lingual inputs?

I also describe things and that also works quite well.

But for a time, boundaries will be pushed that require more compute and they may be a good service to provide that. The hardware is so expensive I imagine their margins can't be very good though. I'd be interested to see their business plan, because the current version of OpenAI in terms of what it offers doesn't seem to be that compelling when extrapolated out 5 years without some other innovative products.

I honestly think Apple will dominate the personal AI angle once they get there. What's left is business and that will be more competitive.

That might be a bit different for big companies where they want to run their own models.

The other factor is that Google and others are certainly not going to sit still. There is no reason to believe that someone as resourceful as Google cannot come up with something as good as ChatGPT, if not better. Companies like Meta are playing the open source card, so they will be the first to benefit directly from the community. So the market will change, but dramatically. It's far too early to bet on any of them (or none of them). My approach is to diversify, wait and see.

There is definitely value here, I use the product a lot myself, but I don't agree that the value is as high as the majority of people seem to think (ChatGPT is going to reshape economies, every industry will replace 90% of humans with some form of AI soon, in more extreme cases that AGI is close to happening, etc...)

I wanted to see if anyone here had examples or use cases that could make me think otherwise

"Computers are useless. They can only give answers" - Pablo Picasso