Hands On – new Apple Intelligence Video – sports uses

Apple is under investor pressure to get its AI features working. It has just posted a 6-minute video indicating features you can use today. I’ve used some of them before, and they were underwhelming on the whole, but sometimes useful and sometimes useless for my needs.

I investigated the current state of play today and have summarised my sports-focused use of them below, along with a link to Apple’s original video.

Here are the eight areas

1. Photo Clean Up
   
2. Genmoji
   
3. Image Playground
   
4. Mail Summarize
   
5. Writing Tools
   
6. Writing Tools with Chatgpt
7. Visual Intelligence with Google Lens
   
8. Visual Intelligence with Chatgpt

Photo Clean Up

I admit to using this feature a LOT. Our family photos number into the thousands, and when there’s a boring program on the TV that I’ve been told to watch by my partner, I tidy up some photos on my iPhone as they scroll through Facebook rubbish on their iPhone.

Here is an example of what you can do. In this photo of two people, I edited out the person to the left in the iPhone Photo app’s Clean Up section. Trace around the object to highlight it and tap to remove. The feature works well in about 7 out of 10 cases, which I think is pretty good. I was going to edit out Ray humorously, but didn’t want to risk him not seeing the funny side, so I will leave the star on show. I also edited out some of the trees on the horizon, and you can see various aberrations where an object has been removed. You often get perfect results, much better than those shown here, it depends on the complexity of the hidden background that the AI has to imagine.

From memory, there are sometimes restrictions for images taken from non-Apple devices. To ease writing this post, I took a photo of my PC screen with Ben Greenfield’s image on Pinterest and created a duplicate.

Rating: 9/10. It’s not perfect, but it’s a useful waste of time.

Genmoji

Genmoji creates a custom emoji based on words you type. Instead, you could type those words, I suppose. That would be quicker, but maybe less fun.

At least this way, I got my first medal of the season as evidenced by the emoji, below.

Rating: 4/10. It’s a well-executed feature. Surely it’s designed for kids? Maybe I’m becoming bitter, twisted, old and grumpy?

Image Playground

Download the free Image Playground app from Apple.

I played with this when it was launched and deleted it about 20 minutes later. I re-downloaded it for this article and will delete it soon after.

You can create an image based on words and a person’s image. Here’s what I made from Run, Sun and Swim. A cat somehow got into the picture rather than the selfie I intended.

Apple suggests using Playground to create themed images for kids’ birthday parties. I couldn’t think of any other uses.

Rating: 1/10. It’s very clever. I’m not sure Apple shareholders will be overly pleased by how much money it took to create this.

Mail Summarise

My partner uses Apple Mail, I don’t. However, I hear good things about the new Mail summarise feature.

You know the people who like the sound of their own voice? Many of them also dump all the contents of their brains into emails they send you. There’s usually something useful somewhere, but finding it is a day-long activity. Mail Summarise does what it says on the tin.

Rating: ? I don’t know, I’ve not used it. It looks like a decent feature. I don’t get many emails that are too long. Even the longer ones I can scan in seconds to find and read the best part, though I would freely admit that means I sometimes miss a key point. I use other excellent tools to summarise all types of information. I might buy a Mac mini soon, if so, I will use Apple Mail for my email, and I suppose I will use this feature.

Mail Rewrite

This is a helpful feature if you use Mail. I would like to know how it works for complex non-US English writers from someone who has used it for a few months.

I don’t use it.

I DO use Grammarly integrated into Microsoft Outlook, and it has transformed how I write emails. So, similar tools like Grammarly will invariably help most people who have yet to try them. Give Grammarly a go, or give Mail Rewrite a go.

Even in Apple’s silly example, where the person used all caps (that’s me when I don’t look at the screen!), Mail Rewrite adds an easy way to correct capitalisation and other text to reflect your intended mood.

Writing Tools

You can use Apple Notes to create a training plan using Apple’s integration of OpenAI’s GPT-4o capabilities.. A year ago, I tried a similar thing with chatGPT when it was first hyped, producing seemingly good results.

Here, I quickly got a home-based, twice-weekly 30-minute plan for a runner who wants to do bodyweight, supportive exercises. It seems superficially sensible and could be refined further with a few more tweaks via the AI.

I suspect those devoting many hours a week to compete on some level will not trust these tools. Perhaps they are OK with other people who want to ‘exercise’ rather than ‘train’ if you get the difference.

Against that argument, we could look at Zwift’s workouts and plans, which are not as good as those from other dedicated training platforms. Lots of people trust Zwift.

We could look at Strava’s recent addition of Runna and think that is a good move, but Runna/Strava cannot include HRV-based readiness as Garmin might. Thus, there are many existing issues with the tools we use. As I said in the previous paragraph, the decision to use guided training must come down to trust.

Rating 8/10: chatGPT is an impressive piece of software with well-publicised issues about inventing facts.

Visual Intelligence

This is a potentially powerful tool, let down by getting it to work. I had to jump through hoops on the iPhone 15 Pro even to access this feature for the first time. When I could access it, I found it hard to lock onto the information/image I was trying to learn more about. That said, I’ve got it working and will probably use it again.

I pointed it at the cactus in my room – that was nicely identified. Then I pulled up some information on my PC screen about the Eton sprints weekend – the triathlons at Dorney Lake. Holding the camera (visual intelligence) at just the right angle and then capturing the event was tricky. Once I managed the first, fundamental step, I was impressed with how it added the event to my schedule. You can use this to scan flyers you might see as you walk in the street, or simply any object you might be curious about. The tools seem to use Google Lens/Image Search behind the scenes.

You can also take a photo and ask Chatgpt about it. As you can see, I lined up various supplements and asked how to use them for race day. The answers given were mainly inadequate.

The products that could be seen are

• Beetroot juice shot – correctly identified and its usage – answer to take 2.5-4 hours before the start of the race
• CurraNz – it got the right answer
• High 5 – it got the wrong name but the right advice
• PhD – eh? What was this? I don’t have such a thing.
• Echinacea – should have said it was useless.
• Tribe Chia bar – should have said to eat it 20-30 minutes before it was needed, and how long it would last, perhaps another 30-40 minutes.

Rating: 6/10: Amazing potential in a handy, handheld format (phone!) but lacking in execution

Take Out

Apple correctly labels its AI as beta, but is under investor pressure to catch up with the competition. If it’s any consolation, Apple is way ahead of Garmin’s poor attempts at AI so far.

The features are aimed at a wide variety of Apple users. Whilst I am not interested in Genmoji and Image Playground, I’m sure some kids somewhere will love the features, and Apple is right to aim features at its full ownership base.

The remaining writing, image and visual intelligence tools are very promising and, who knows, I may use them all quite a lot in the years to come. I see nothing here that comes close to replacing humans, but I do see specific solutions that automate aspects of our daily lives, and that is ENTIRELY what tech should be doing, IMHO.