Slight problem being you don't hear through your eyes.

For VR, since you have to put the helmet in your head, it is usually assumed users don't mind putting headphones as well, so most solutions are via headphone-based spatialization.

Good point, piezos do also generate voltages when deformed, so this could conceivably be run in reverse... Microphone arrays are already used for directional sound detection, so this could have fun implications for that application given the claimed density of microphones.

Technically yes, piezo cells are reversible, just like about anything that can be used to emit sound. You can use the array for programmable directionality as well.

A proper telescreen is not the one you watch and listen to, but the one that watches you and listens to you, %little_brother%

Theoretically, any frequency can be produced by the interference of ultrasonic waves, but the amplitude is questionable, given that these emitters are embedded into a thin substrate.

Audi somewhat did this with the display in the Q8 E-Tron. I don’t recall if the display is OLED, but it has excellent haptics.

A laptop improvement I'd love would be a haptic keyboard. You could then make the entire laptop waterproof as well as do keyboard reconfiguration for certain tasks.

I always though Apple's Touch Bar was some kind of entry point into that idea but it wasn't, wasn't useful, and just died.

Localization of sound is primarily based on the time difference between the ears. Localization is also pretty precise, to within a few degrees under good conditions.

The main utility isn't for the user to more precisely locate the sound source within the screen. Phased speaker arrays allow emitting sound in controlled directions, even multiple sound channels to different directions at the same time.

I'm sorry, but this is not accurate at all. Using "only" two signals, humans are quite good at localizing sound sources in some directions:

Concerning absolute localization, in frontal position, peak accuracy is observed at 1÷2 degrees for localization in the horizontal plane and 3÷4 degrees for localization in the vertical plane (Makous and Middlebrooks, 1990; Grothe et al., 2010; Tabry et al., 2013).

from https://www.frontiersin.org/journals/psychology/articles/10....

Humans are quite good at estimating distance too, inside rooms.

Humans use 3 cues for localization, time differences, amplitude differences and spectral cues from outer ears, head, torso, etc. They also use slight head movements to disambiguate sources where the signal differences would be the same (front and back, for instance).

I do agree that humans would not perceive the location difference between two pixels next to each other.

You are misinformed.

Amplitude, spectral, and timing are all integrated into a positional / distance probability mapping. Humans can estimate the vector of a sound by about 2 degrees horizontal and 4 degrees vertical. Distance is also pretty accurate, especially in a room where direct and reflected sounds will arrive at different times, creating interference patterns.

The brain processes audio in a way not too dissimilar from the way that medical imaging scanners can use a small number of sensors to develop a detailed 3d image.

In a perfectly dark room, you can feel large objects by the void they make in the acoustic space of ambient noise and reflected sounds from your own body.

Interestingly, the shape of the ear is such that different phase shifts occur for front and rear positions of reflected and conducted sounds, further improving localization.

We often underestimate the information richness of the sonic sensome, as most live in a culture that deeply favors the visual environment, but some subcultures and also indigenous cultures have learned to more fully explore those sensory spaces.

People of the extreme northern latitudes may spend a much larger percentage of their waking hours in darkness or overwhelming white environments and learn to rely more on sound to sense their surroundings.

Good find - the first video is a frequency sweep, video 2 has some music.

Edit - I'm not sure that's the same thing? The release talks about pixel based sound, the linked paper is about sticking an array of piezoelectric speakers to the back of a display.

edit 2- You're right, the press release is pretty poor at explaining this though. It is not the pixels emitting the sound. It's an array of something like 25 speakers arranged like pixels.

https://www.eurekalert.org/news-releases/1084704

This technology literally sounds like a terrible idea. Why do I want to hear one drummer's drum over the others?

https://advanced.onlinelibrary.wiley.com/action/downloadSupp...

These are a thing: https://hackaday.com/2019/10/26/building-the-worlds-best-dml...

You could always have a separate speaker for low frequencies. Low frequency sound tends to be perceived with less of a directional component, so if you have nice directionality of the highs and a single source for the lows, that's pretty acceptable. Removing the need to handle highs might make it easier to put together a speaker for the lows in the confines of a modern flat screen.

Surely? Images play at <120Hz and sound requires you to run at or above the nyquist limit which for human hearing has the standard of 48kHz.

So all AV systems run image and sound separately and thus you can affect them separately.

(I assumed you assumed the changing colors displayed on the pixels with 120Hz somehow drive the sound which needs to change at 48000Hz)

Anything but separate inputs makes no sense giving the magnitudes different frequencies at which it needs to be driven. Also, the color/brightness of a pixel does mean nothing for the sound.

Yeah, I don’t know, I suspect you’re right. I know that 25 years ago there were big flat speakers thay drove sound with little holes. Those things could only handle high ends despite being very large themselves. But I know that with DSP you can treat an array of microphones as a single microphone, and the diameter of the array sort of dictates the size of the microphone. Speakers and mics tend to sort of be opposites, in that you can drive or be driven by either. For years I’ve wanted to build a wall of cheap speakers and experiment with this. I suspect I’d find out what we already know, which is to get deep frequencies you have to move a lot of air, and an array of small piezos can never move a lot of air…

That said, it might matter a lot what the substrait was. If it was light and flexible, you could maybe get all the piezos to move it in a way to get a very deep frequency. You could probably get deeper frequncies than the power output of the piezos by taking advantage of the resonance frequency of the material. But you’d be stuck with that one frequency, and there’d be a tradeoff in response time

I've got a Sony Bravia with similar technology for a few years now (it uses small actuators instead of generating from the oled itself, but the vibrations will be the same) and it's still sounding fantastic after daily use and a couple of moves.

Because audio runs at such low sample rates, it's not exorbitant by current standards. Suppose you have an 8×16 array of speakers behind your screen, each running at 96ksps. That's only 12.3 million samples per second to generate, on the order of 200 million multiply-accumulates per second for even the most extreme scenarios. Lattice's iCE40UP5K FPGA https://www.farnell.com/datasheets/3215488.pdf#page=10 contains 8 "sysDSP" blocks which can do two 8-bit multiply-accumulates per clock cycle at 50MHz even when pipelining is disabled, so 800 million per second. It's 2.1 by 2.5 mm and costs US$5 at Digi-Key. I'm not familiar with AVX, but I believe your four-core 2GHz AVX512 CPU can do 256 8-bit multiply-accumulates per cycle, five hundred thousand million per second, so we're talking about an exorbitant amount of computation that's 0.04% of your CPU.

Probably similar to antennas — using phase shifting and interference.

Right.

It's a way of putting speakers behind a display, which will probably be useful. This may improve bass response in laptops, because the speaker area is larger.

Not this but similar, Sony’s acoustic surface audio for their displays uses actuators behind the screen to vibrate it. it’s not crazy bass but I’d say it’s on par with regular built in tv/laptop speakers

If you get enough, you might be able to do some really interesting things using it as a phased array

Not niche at all. You could have a phone for example that plays sounds from areas of the screen where they originate. Key presses, buttons, notification pop ups, etc.

I think you're trying to make an argument similar to those arguing against "retina" displays, i.e. there is some minimum perceptual angular resolution for sound, so information finer than that is pointless? I think you're either underestimating the perceptual resolution or assuming a very small screen at a large distance.

I think that kind of resolution is good enough to overlap a lot of task focused screen fields of view. I have experienced a pretty clear "central" sound stage within 30-45 degrees or so with regular stereo speakers. That field can imply a lot of subtle positioning within it, not even considering wild panning mixes. I'm talking about the kind of realistic recording where it feels like a band in front of you with different instruments near each other but not colocated, like an acoustic ensemble. Obviously you cannot shrink this down to a phone at arm's length or a small embedded control screen and still have the same amount of spatial resolution crammed into a very narrow field.

When I sit with a 14" laptop at a normal distance for typing, it is also easy to distinguish tapping sounds along the screen. I just did a blind test by closing my eyes and having my wife tap the screen with a plastic pen. The interesting question to me, though, is whether that is just perception via the binaural sense, or really incorporating some intuition about the screen's structure. It's response to a tap does clearly vary by distance from bezels and hinges...

Interesting, I bought a parametric speaker (from Kickstarter of all places [1]), it too uses piezoelectric emitters.

My understanding: DML speakers directly emit audible sound by vibrating at the resonance frequency of a material. Parametric speakers emit ultrasonic waves and control how those waves interfere using phased-array techniques, the interference is what produces sound.

It produces completely different results compared to DML: The sound quality is not good, interference just can't produce the full spectrum of audible frequencies.

In return, you gain extreme control over directionality: the interference only happens in midair or when the ultrasonic waves hits something solid, the speaker itself is mostly silent. It's actually "too directional" for me to use, the beam is still audible after multiple bounces off of solid objects: I was hoping to use it in a office as a sort of "personal speaker", but after the beam strikes my head and I hear sound, a part of it is reflected and bounces off the ceiling and floor repeatedly, causing the speaker to be audible at the opposite end of the floor.

[1] https://www.kickstarter.com/projects/1252022192/focusound-th...