On this put up, I evaluate ChatGPT and Claude on an Alaskan cruise. Claude appears higher at dealing with lengthy content material, and ChatGPT shorter content material. Utilizing each chatbots, I requested many inquiries to find out about my cruise environment. The chatbots expanded my curiosity and made me extra attentive to my surroundings by encouraging limitless questions.
At the moment, two of the preferred chatbots are ChatGPT and Claude. Most individuals have heard of ChatGPT, fewer Claude. Till not too long ago, Claude was obtainable totally free in a restricted method, permitting solely a sure variety of queries per hour. Then Claude got here out with Claude Professional, providing 5x the quantity of information queries. They priced it the identical as ChatGPT: $20/month.
I’ve subscriptions to each ChatGPT Plus and Claude Professional and have been evaluating them. Each present equally clever solutions to most questions I ask. The chatbots principally differ within the enter supply limits. Claude has a 100k enter supply restrict, whereas ChatGPT Plus’s enter supply is round 4k tokens. (100k tokens is about 75k phrases. 4k tokens is about 3k phrases.)
This enter supply additionally pertains to the context window of the chat session. This implies you’ll be able to add a protracted article and observe it up with a number of questions (for instance, a 90k token-length article adopted by 10k token-length chat), or you’ll be able to add a brief article and observe it up with lots of questions (for instance, 10k token-length article adopted by 90k token-length chat).
For instance, after I add a number of chapters from my API documentation course into Claude, I can solely ask a number of questions within the chat session earlier than I’ve to start out a brand new chat session. And with a brand new chat session, the context of the uploads is misplaced. However even having this add restrict measurement is phenomenal in comparison with ChatGPT, whose token restrict is round 4k.
Anyway, listed here are some further variations between Claude and ChatGPT:
- You’ll be able to add PDFs to Claude (max file measurement is 10 MB). With ChatGPT, it’s a must to paste within the content material.
- Claude appears quicker with its responses. The location masses shortly, and solutions seem extra quickly. In distinction, ChatGPT typically hangs and requires me to hit Regenerate. At occasions ChatGPT’s textual content seems word-by-word at a plodding tempo.
- Claude has extra safeguards (equivalent to guarding in opposition to biases). In distinction, you’ll be able to “jailbreak” ChatGPT extra simply.
I’m certain there are much more nuances throughout totally different data domains. For instance, is one higher at math than one other? Coding questions? Specialised data? I’m unsure. That sort of testing is past the scope of my put up.
Different realizations about LLMs typically
Earlier than leaping into extra comparability checks between Claude and ChatGPT, let me detour a bit. My worth as an off-the-cuff tech commentator is in relating my very own experiences with AI, so bear with me.
Regardless of my enthusiasm for AI, my spouse stays uninterested, skeptical, and dismissive of AI chatbots. I hold ready for her to have that magic second the place she will get it, however thus far, it hasn’t come. My speculation for her apathy is that LLMs are seemingly solely interesting to creator varieties — content material creators, coders, builders, and so on. In lots of different situations, AIs don’t have a compelling use case.
However this previous week I had alternatives to make use of Claude and ChatGPT various occasions in a non-creator situations. This week I used to be on a cruise to Alaska, spending time touring alongside the western coast of Canada and stopping in Juneau, Skagway, and Ketchikan. In the course of the cruise, we listened to naturalist lectures and tour information explanations and tales. With every lecture and tour, I had questions I wished to ask, like most individuals. Generally I requested the information, however I additionally began asking Claude and ChatGPT.
For instance, through the ship naturalist’s lectures on bears, salmon, and bushes, I requested Claude the next:
- Who’s Robert Raincock, the naturalist? [This was the ship’s lecturer.]
- What questions would particularly join with Raincock’s analysis in a private method?
- I don’t appear to essentially care about whales or seeing them. Is one thing improper with me?
- What does it imply for salmon to be anadromous?
- What does it imply for salmon to spawn?
- When individuals catch salmon, are the salmon principally within the spawning stage?
- What does this saying check with: if black combat again, if brown lay down
- How might I have interaction with a grizzly in hand to claw fight and win? … What if I’ve a protracted knife? … Even when stabbed within the eye?
- How many individuals in Alaska die from bear assaults yearly?
- What do bears love that people additionally love?
- Do individuals eat bear meat?
- Will local weather change drive polar bears south?
- If polar bear meets a grizzly, will they combat?
- Why is a fed bear a useless bear?
- Are some other fish apart from salmon anadromous?
- How are AI instruments relevant to bear or salmon examine?
- What are the important thing concepts within the e book Salmon within the Timber?
- How do salmon know once they’ve reached the purpose the place they started life? … What are the organic theories about how they do that?
- Why don’t extra animals eat the useless salmon which have simply completed spawning and are laying within the water?
- Which is the preferred concept for the way salmon navigate again residence?
- Do all salmon originate in Alaska?
You’ll be able to see my Claude chat about salmon, bears, and bushes right here.
As I began to ask a number of preliminary questions, the solutions to these questions gave rise to extra. My curiosity grew and grew. I quickly grew to become filled with questions.
In the course of the naturalist lectures, the speaker didn’t take questions (there have been 200 individuals listening). However even when he did, are you able to think about somebody asking 20 annoying questions like this? Not solely wouldn’t it derail the lecture’s focus, a few of my questions are embarrassing. Do I actually need to ask questions on methods to win at hand-to-claw fight in opposition to a bear? Or admit that I don’t actually care about seeing whales, or that I don’t know what “salmon spawning” means or whether or not all salmon originate from Alaska? Nonetheless, we will ask the dumb or embarrassing inquiries to AI!
Additionally, AI instruments are good at answering common data questions like this. You can in all probability discover most of those solutions on Wikipedia, however solely after a protracted and tedious search. For instance, attempt searching the outcomes to my query “Are some other fish apart from salmon anadromous?” on Wikipedia. The solutions are scattered in fragments throughout numerous pages. It is likely to be tougher to search out solutions to questions on knife ways in Grizzly fights, besides maybe on Reddit threads.
In distinction, LLMs present fast, quick solutions straight addressing my query. Might the solutions be riddled with errors? Certain. And when you’ve requested a query that you simply don’t know the reply to, you’re ready of vulnerability. Nonetheless, I’ll gladly take fast solutions which can be 90% correct over 99% correct solutions that take 10x as lengthy to search out.
I’ll usually pose the identical query to each Claude and ChatGPT as a validation test on one another. I assume the widespread floor of their responses is much less more likely to be a hallucination. It’s like getting a second opinion from one other physician. When a number of individuals inform you an identical factor, the probabilities of it being true improve.
Do LLMs encourage curiosity?
In the course of the cruise excursions, I discovered myself pondering, I’ve a tool in my pocket that may present solutions to virtually any questions I’ve. As such, I began utilizing Claude and ChatGPT increasingly on the journey, in virtually each place I visited.
On a naturalist tour by way of a temperate rainforest space, I did ask our tour information (reasonably than the AI chatbots) a few questions, like what are these vegetation which have lengthy leaves (skunk cabbage, it seems), what sort of fish are within the lake (rainbow fish), why are the bushes round us withering (as a consequence of tannic acid), and the way do they calculate the age of bushes with out chopping them all the way down to rely the rings (they take core samples).
These questions have been warmups for the bigger questions that began to floor in my thoughts. Questions like, what did this forest seem like 300 million years in the past? When did bushes initially type? If the lifetime of a tree might be 900+ years, do bushes have a slower tempo of evolution? If bushes evolve so slowly, and evolution is such a key trait in the direction of dominance, why are bushes nonetheless so plentiful? Should you might hint again the evolution of bushes to the primary tree, which type wouldn’t it be? Did Spruce bushes evolve from Western Hemlock, or vice versa? What was the primary tree? Which organisms evolve the quickest and slowest? Did tree life must evolve first with a purpose to assist animal life?
Since our information wasn’t a full-fledged biologist, I didn’t ask him these questions. Plus, the solutions I used to be getting from Claude (when service was obtainable) have been fairly good. At this level, my questions would have solely examined his data. A part of me wished to ask the inquiries to the information as a result of I used to be happy with how intelligent they have been, particularly my query in regards to the first tree, or what the forest appeared like 300 million years in the past. However wanting intelligent didn’t appear purposeful, and there wasn’t time for limitless Q&A on this brief rainforest jaunt anyway.
These situations with LLMs have been fairly cool. I began to get assured that I might discover solutions to just about each query I had. I felt extremely succesful. What do I do with this infinite, quick details about the whole lot and something?
If nothing else, asking inquiries to AI instruments begat extra questions, and that limitless stream of questions made me extra curious, observant, and attentive. A semi-quiet, stunning forest was now not only a stunning temperate forest setting. It grew to become an area for essential inquiry, for inquiries to reply. Every reply led to extra questions and ideas. By way of these instruments, my curiosity expanded, my consideration centered and stayed current, and I used to be by no means bored.
Coming again to an earlier theme: are LLMs simply good for creators? In these vacationer situations, clearly not. They’re helpful in unfamiliar settings, so long as you’ve a curious perspective. Ask one query and also you’ll end up asking half a dozen extra. Will these instruments catch on and develop into instruments utilized by the mainstream? I’m unsure. The default search instruments will morph increasingly into LLMs. Possibly some day, we’ll use AI chat as the first interface to the world’s data.
How will we use this data to not solely reply fundamental questions like mine, however to maneuver in the direction of new questions that don’t have solutions? Will these instruments gas and broaden our curiosity? How will we synthesize the solutions into new data that’s outdoors the LLM’s predictive attain?
Evaluating Claude’s solutions with ChatGPT
Let’s get again to evaluating Claude versus ChatGPT. It simply so occurs that, throughout one of many ship excursions, we have been in an tour group (ziplines!) with the ship’s navigator. The navigator invited us on a particular tour of the ship’s bridge, the place we might see the captain, officers, and different crew utilizing the navigation controls. To organize for this particular bridge tour, I began pondering up inquiries to ask:
- Navigation. How do you navigate the ship? GPS? What if GPS goes down? Satellite tv for pc GPS? Should you needed to navigate utilizing one other technique, what wouldn’t it be? Might you navigate by means of the celebrities?
- Avoiding hazards. How do you see underwater to ensure we don’t run into something like icebergs, whales, shallow water, massive rocks, different stuff within the ocean? How do you keep away from collisions with different cruise ships within the space? How a lot do you take note of climate in route planning and navigation?
- Automation. Does the map information get delivered robotically to parts of the ship, or does the crew right here do all of the navigation manually? How are AI instruments utilized in navigation? Autorouting? Auto-parking? Dangerous climate avoidance? Collision avoidance?
- Metropolis welcomings. Are cruise ships restricted to the locations they’ll cease because of the necessities of parking the ships? Are there any cities that don’t permit cruise ships to port? Do cities love or hate it when a large cruise ship pulls into port?
- Behind the scenes maneuvers. What are some issues passengers didn’t even find out about on this journey which may have been security hazards we needed to keep away from or altered selections made about our route?
- Leaving passengers behind. What occurs if a passenger doesn’t come again from an on-shore go to? How lengthy do you wait? How ceaselessly does it occur?
- Sinking situations. What’s the most definitely state of affairs that will trigger the ship to sink? A tsunami from an erupting volcano? Terrorist takes management? Bilge pump malfunctions? Rogue waves that capsize the ship? Fireplace from an engine explosion?
- Analytics from wearable tech. Can you utilize the medallions to trace the preferred actions on the ship after which cater these actions round what individuals do probably the most? E.g., tons of metrics round individuals going to a sure present vs. not others? Can these analytics be used to raised form the actions and cruise experiences?
- Crew chiefs. Is there somebody in control of the passengers total? Is the captain principally involved in regards to the ship whereas there’s like a head individual over the passengers?
I did leverage the AI chatbots to assist give you among the questions.
I initially deliberate to ask my inquiries to the bridge crew. However then I made a decision to ask Claude and ChatGPT for the solutions even earlier than the tour. Right here’s my preliminary immediate for the chat session:
You are the captain of the Princess cruise line, a luxurious ship that carries 9,000 individuals from Seattle to Alaska, stopping at Juneau, Skagway, and Ketchikan. You are giving a tour to a choose subgroup of passengers. Reply these questions:
Then I went by way of every of the above bullets, pasting them in individually reasonably than suddenly as a result of the solutions are typically extra detailed that method. You’ll be able to view the responses right here:
Trying on the responses, ChatGPT’s solutions are extra detailed and thorough. Claude’s solutions are first rate however shorter and extra succinct. Throughout ChatGPT’s responses, the chat session timed out twice, requiring me to hit Regenerate for the response. Moreover, ChatGPT took considerably longer to kind out the solutions than Claude.
In the course of the precise bridge session (see pic under), I requested solely a few questions. Claude and ChatGPT’s responses didn’t actually predict what the navigator defined on the bridge intimately. However it was nonetheless enjoyable to do that.
In conclusion, Claude is the higher alternative when you’ve longer content material to investigate, however ChatGPT is likely to be higher for shorter content material. Each instruments, nonetheless, can broaden your curiosity and fill your thoughts with questions. This curiosity in regards to the world round you may make you extra attentive and current.
About Tom Johnson
I am an API technical author based mostly within the Seattle space. On this weblog, I write about matters associated to technical writing and communication — equivalent to software program documentation, API documentation, AI, data structure, content material technique, writing processes, plain language, tech comm careers, and extra. Take a look at my API documentation course when you’re on the lookout for extra information about documenting APIs. Or see my posts on AI and AI course part for extra on the most recent in AI and tech comm.
Should you’re a technical author and need to carry on prime of the most recent developments within the tech comm, make sure to subscribe to e mail updates under. You can even study extra about me or contact me. Lastly, notice that the opinions I categorical on my weblog are my very own factors of view, not that of my employer.