Apple Pencil Used as Weight Scale, Synthesizer and 3D Image Editing Controller

[MacRumors]

While the Apple Pencil is designed as a sketching tool for creative professionals, MacRumors reader Simon Gladman has created three Swift demo apps that show the accessory being used for three unconventional purposes -- as a weight scale, controlled synthesizer and 3D controller for image editing.

PencilScale

PencilScale, based on Goodman's Plum-O-Meter, is an experimental app that uses a homemade harness to turn the Apple Pencil into an electronic scale that is highly sensitive, but not incredibly accurate.

The experiment works by subtracting the touch's force from a base weight, which is "set as the current touch force when the 'zero' button is pressed," and multiplying it by 140 for a very rough weight in grams.

PencilSynth

PencilSynth is an AudioKit-powered synthesizer that can be controlled by the Apple Pencil depending on its orientation and position.
Apple Pencil's horizontal position on the screen controls frequency
Apple Pencil's vertical position on the screen controls the modulating multiplier
Apple Pencil's altitude angle controls the carrier multiplier
Apple Pencil's azimuth angle controls the modulation index
PencilController

PencilController is an experimental image processing app that uses the Apple Pencil as a controller for the fine setting of parameters on Core Image filters.

The demo has three image filtering modes:
Hue/Saturation - Apple Pencil's azimuth angle controls hue and its altitude angle controls the saturation
Brightness/Contrast - Apple Pencil's altitude angle along North/South controls contrast and the angle along West/East controls brightness
Gamma/Exposure - Apple Pencil's altitude angle along North/South controls exposure and the angle along West/East controls gamma
Gladman explains that "the app uses a spring loaded pattern, so the user needs to hold down one of the mode keys in the bottom left of the screen to stay in the filtering mode."

The source code for all three projects is available on GitHub.

Article Link: Apple Pencil Used as Weight Scale, Synthesizer and 3D Image Editing Controller

 

[joueboy]

I know this is sounds cool. But come on guys I'm tired of hearing this non-sense use of technology. There is abviously a much better device for such thing and it doesn't cost a lot of money to have one. I'm talking about the scale and somebody did it on iPhone 6S 3D Touch too. It's because you can it doesn't mean you have too.

 

[Xprmntr]

The synth is a pretty cool concept though, games would be interesting too

 

[nagromme]

Think what you could do with 3 pencils! You could control that synthesizer with one, and jam the other 2 in your ears until the racket stops.

 

[teslo]

I know this is sounds cool. But come on guys I'm tired of hearing this non-sense use of technology. There is abviously a much better device for such thing and it doesn't cost a lot of money to have one. I'm talking about the scale and somebody did it on iPhone 6S 3D Touch too. It's because you can it doesn't mean you have too.

yeah, why would anyone ever want to do things?

i, for one, am witnessing exactly what i'd hope would start happening upon the release of a new device: tinkering, feedback, and constructive criticism. i'm sorry you're tired, i hear energy drinks are all the rage nowadays.

 

[Waxhead138]

Aren't the phrases "Highly Sensitive" and "Not Incredibly Accurate" a bad combination under pretty much any circumstance?

.....As a side note, those two phrases remind me of almost every girlfriend I've ever had, the first phrase typically causing the second....

 

[blackcrayon]

I know this is sounds cool. But come on guys I'm tired of hearing this non-sense use of technology. There is abviously a much better device for such thing and it doesn't cost a lot of money to have one. I'm talking about the scale and somebody did it on iPhone 6S 3D Touch too. It's because you can it doesn't mean you have too.

Safety issues (of the device) aside, I thought the iPhone 6s scale was pretty clever and potentially useful. Just like anything else, the scale you have with you is a lot more useful than the one you don't

 

[ApfelKuchen]

I know this is sounds cool. But come on guys I'm tired of hearing this non-sense use of technology. There is abviously a much better device for such thing and it doesn't cost a lot of money to have one. I'm talking about the scale and somebody did it on iPhone 6S 3D Touch too. It's because you can it doesn't mean you have too.

"Non-sense?" I think there are some shades of gray here.

I don't see people buying the Pencil to use it as a scale - it requires calibration and additional hardware - it's nothing more than a fun proof-of-concept. I'm also not sure that it's a substantial improvement over manipulating sliders in a graphics app - it seems far more compelling for other graphics-related uses (drawing, or creating masks in brush/retouch modes). But do not underestimate the appeal (to a musician) of moving a physical object (baton) to control sound. That is what they do, and the Pencil definitely offers a greater opportunity for nuance.

So, for 'A' - some may get additional use from an existing purchase. For 'B' - those who purchased it for a specific purpose may get greater-than-anticipated usage for that purpose. For 'C' - people will buy it for that specific purpose (although not likely in great numbers - the Theremin never found its way to the elementary school orchestra, either).

The takeaway here is, for the most part, "Here are more ways to use something I already paid for." It's all about multi-tasking. It's the reason smart phones took over from feature phones. You already have it in your pocket, and apps then give you more ways to get use from the thing. That's nothing to scoff at.

 

[groovyd]

Think what you could do with 3 pencils! You could control that synthesizer with one, and jam the other 2 in your ears until the racket stops.

does the iPad Pro support multiple pencils at once? This would open the door for a ton of very clever apps besides enabling the possibility of a scale with very high precision. I agree though just get a scale if that is what you need.

 

[Makosuke]

Aren't the phrases "Highly Sensitive" and "Not Incredibly Accurate" a bad combination under pretty much any circumstance?

Nope, from a scientific terminology perspective, that actually makes perfect sense.

Accuracy, scientifically speaking, is how close to the real value a measurement will produce.

Precision (aka sensitivity) is how small of a quantity you can measure, accurately or otherwise.

And repeatability is how close to the same value taking the same measurement at two different times will produce, accurately, precisely, or otherwise.

So: An accurate scale will tell you something weighs 502 grams when it weighs 502 grams. A precise scale will tell you that something weighs 498.93 grams, although it might actually weigh 502.00 grams if the scale is not also accurate. And a repeatable (and precise) scale will tell you that something weighs 498.93 grams every time you weigh it, although if it's not also accurate it might really weigh 502 grams.

This particular "scale", silly or otherwise, will give you a lot of digits of precision on the weight measurement, so it's quite precise. It however isn't calibrated at all, so the accuracy is bad. No idea whether it's repeatable or not.

 

[carrrrrlos]

What’s the fascination behind making iOS devices into somewhat accurate scales?

 

[JustThinkin']

I know this is sounds cool. But come on guys I'm tired of hearing this non-sense use of technology. There is abviously a much better device for such thing and it doesn't cost a lot of money to have one. I'm talking about the scale and somebody did it on iPhone 6S 3D Touch too. It's because you can it doesn't mean you have too.

Let me plead with you to never work for Apple or take any job that requires the least bit of creativity!

 

[garirry]

I know this is sounds cool. But come on guys I'm tired of hearing this non-sense use of technology. There is abviously a much better device for such thing and it doesn't cost a lot of money to have one. I'm talking about the scale and somebody did it on iPhone 6S 3D Touch too. It's because you can it doesn't mean you have too.

Jesus ****ing christ, it's called EXPERIMENTATION. People EXPERIMENT with various devices and tools to create fun stuff. Maybe in real life it's incredibly pointless (there are better devices/software for cheap), but from a pure technical standpoint, it's very interesting.

 

[FlexMonkey]

I know this is sounds cool. But come on guys I'm tired of hearing this non-sense use of technology. There is abviously a much better device for such thing and it doesn't cost a lot of money to have one. I'm talking about the scale and somebody did it on iPhone 6S 3D Touch too. It's because you can it doesn't mean you have too.

Hi!

I'm actually the same guy that did the "Plum-O-Meter" for weighing things on an iPhone

Of course there are better devices, but I'm interested in experimenting, tinkering and playing then seeing what results. Actually, using the Pencil as a controller works really well for the fine adjustments in the image processing app and the fact the it has five degrees of freedom makes it a unique input device for the audio app.

Without some left field thinking, we'd all be using abacuses!

Cheers!

Simon

 

[2457282]

I know this is sounds cool. But come on guys I'm tired of hearing this non-sense use of technology. There is abviously a much better device for such thing and it doesn't cost a lot of money to have one. I'm talking about the scale and somebody did it on iPhone 6S 3D Touch too. It's because you can it doesn't mean you have too.

I agree. Instead of developing these silly things, we should be working on spelling and grammar checkers. Those are the more immediate need compared to the silly games. Wouldn't you agree?

 

[Alenore]

I know this is sounds cool. But come on guys I'm tired of hearing this non-sense use of technology. There is abviously a much better device for such thing and it doesn't cost a lot of money to have one. I'm talking about the scale and somebody did it on iPhone 6S 3D Touch too. It's because you can it doesn't mean you have too.

A much better use than distract people and experiments?
Oh.

 

[enric0]

Hi!

I'm actually the same guy that did the "Plum-O-Meter" for weighing things on an iPhone

Of course there are better devices, but I'm interested in experimenting, tinkering and playing then seeing what results. Actually, using the Pencil as a controller works really well for the fine adjustments in the image processing app and the fact the it has five degrees of freedom makes it a unique input device for the audio app.

Without some left field thinking, we'd all be using abacuses!

Cheers!

Simon

Interesting work, never sit down and stop experimenting in computer science as in music and life...

 

[macduke]

I'm gonna buy an iPad Pro to stand on to track my weight on my Apple Watch. Wish me luck!

 

[CFreymarc]

I know this is sounds cool. But come on guys I'm tired of hearing this non-sense use of technology. There is abviously a much better device for such thing and it doesn't cost a lot of money to have one. I'm talking about the scale and somebody did it on iPhone 6S 3D Touch too. It's because you can it doesn't mean you have too.

Welcome to the lunatic fringe of tech. Crazy demos like this are an incubator for new technologies. You must remember what you see as "useless" may have an application in a market you don't even know exists.

The early VR days of the mid-90's had tons of strange stuff like this for consideration. One of my favorites was tracking arm movements to conduct a virtual symphony in an immersive VR space. This was demo'ed at a SigGraph back in the day.

Some said it was useles since modern orchestra leaders were very anti-technology. However this arm tracking demo lead to the student working a lot of paying gigs for of VR chamber apps using these techniques.

 

[CFreymarc]

Jesus ****ing christ, it's called EXPERIMENTATION. People EXPERIMENT with various devices and tools to create fun stuff. Maybe in real life it's incredibly pointless (there are better devices/software for cheap), but from a pure technical standpoint, it's very interesting.

Exactly, demos like this make you think what could be done. It is like a concept car at an auto show. While not an actual product, it is there to show concepts and ideas that are not ready for prime time. Those that scoff at it are just another piece of feedback.

Hi!

I'm actually the same guy that did the "Plum-O-Meter" for weighing things on an iPhone

Of course there are better devices, but I'm interested in experimenting, tinkering and playing then seeing what results. Actually, using the Pencil as a controller works really well for the fine adjustments in the image processing app and the fact the it has five degrees of freedom makes it a unique input device for the audio app.

Without some left field thinking, we'd all be using abacuses!

Cheers!

Simon

Simon, glad to meet you. I'm impressed by your work and have done similar prototype and concept work in different industries. Pick up a soldering iron and start to work below the code -- you can have even more fun with iOS demos! BLE device twiddling is getting very hot these days.

My experience has shown the more a small minority of public is upset or right out angry of your demos, the more you have something. You don't owe anything to critics. Play around with new tech and concentrate on whom has money with a problem that needs solved. Once you provide that and cash in, the critics get quite real fast as they move on to complain elsewhere.

 

[batchtaster]

It's because you can it doesn't mean you have too.

Sometimes it's about possibilities rather than practicalities.

Look at the iPhone spirit levels. Or the iPhone flash torch. They're now built-in features that barely rate a mention because they're taken for granted. In a couple of years, the more evolved, practical offspring of these ideas will be in the same "taken for granted" boat.

 

[SHirsch999]

I know this is sounds cool. But come on guys I'm tired of hearing this non-sense use of technology. There is abviously a much better device for such thing and it doesn't cost a lot of money to have one. I'm talking about the scale and somebody did it on iPhone 6S 3D Touch too. It's because you can it doesn't mean you have too.

As a father of a child with Type 1 Diabetes being able to use my phone as a scale would be incredibly useful. Weighing food and calculating its carbohydrate content is one of the hallmarks of proper insulin dosing. Since food scales tend to be rather difficult to carry around, perfecting this technology would be very appealing.

 

[CFreymarc]

Sometimes it's about possibilities rather than practicalities.

Look at the iPhone spirit levels. Or the iPhone flash torch. They're now built-in features that barely rate a mention because they're taken for granted. In a couple of years, the more evolved, practical offspring of these ideas will be in the same "taken for granted" boat.

Illegitimi non carborundum!

 

[Adithya007]

And the weights fall onto the iPad's screen, Screen cracks, Apple costs 400$ for screen replacement. Bravo!

 

[jm001]

I'm reading several negative feedback on what is happening out there with the introduction of force touch, the pencil and pro. All I can say is this IS what Apple is all about i.e. "Think different". They, Apple, provide the tool and we the public should be thinking outside the box like what we see out there with people tinkering with the 6s and Pro. Sure there's a lot of proof-of-concepts, but remember when the iPad was first introduced? Now look at all the applications it can do. Who knew that your phone can be a point of sale system with a dongle and app taking credit and debit cards. Now you can even receive payment via NFC. I use my phone as a scanner to scan documents and they scan better than my old flatbed. So kudos to "the crazy ones, the misfits, the rebels,... the round pegs in the square holes..."