Facebook Using AI to Describe Photos to the Blind

As any user of Facebook will immediately tell you, photos play a major function in sharing of life stories. From hiking mountains to capturing the beauty of a well-prepared dinner, photos add to the story tremendously. After all, a picture is worth a thousand words, right? (And thus the success of emoticons, I think).

But what if those photos cannot be seen by visually challenged FB friends? Not to worry. Facebook’s engineers have harnessed the power of an artificial intelligence network (AI)  to describe these pictures to blind or partially blind users.

Facebook calls the system “automatic alternative text” and it’s based on a neural network primed with billions of parameters and millions of examples. Such neural networks – vast, complex databases designed to mimic the human brain as closely as possible – are playing an increasingly important role in modern computing.

The AI doesn’t actually “see” what is in the photo, and, as with all things computer at the moment, doesn’t understand context (and context is vital for a true understanding of photos). Rather, the AI compares the objects in the image with its “its vast internal database of similar photos and make an educated guess about what’s being shown.”

As to context, part of the challenge is in getting computers to recognize what’s most important in an image, whether that’s the people, the background, or the “action.” This requires a great more programming, but that’s in the future.

For now, the AI system returns a confidence score indicating how sure it is that it can identify what’s in the picture. If this is above 80 percent, an automatically-generated caption appears. 

facebook-photos-ai-2

When objects and people have been identified, Facebook’s software constructs a sentence to describe the picture, usually ordered by how confident the AI is about the presence of each element. If there’s some ambiguity about the picture then the sentence starts with “image may contain” to express that uncertainty.

The feature is live now in the Facebook iOS app, as long as your language is set to English and you’re in the US, UK, Canada, Australia, or New Zealand. Facebook says it hopes to roll out the service to more platforms, languages and markets in the near future. It actually works with any screen reader software – on iOS you can enable it via the VoiceOver tool in the Accessibility section of Settings (under General), for example.

I wonder what the AI writes about such photos:

 

Call a Doc

So, you have a sore throat but you can’t get in to see a doctor for a few days. What do you do?

If things advance as technology allows, you can phone your doctor or contact her on the Internet, and be attended to virtually. It’s called “telemedicine” and the idea is gaining traction around the world. According to an article in the Wall Street Journal:

“Driven by faster internet connections, ubiquitous smartphones and changing insurance standards, more health providers are turning to electronic communications to do their jobs—and it’s upending the delivery of health care.”

In some cases, it makes sense.

“Doctors are linking up with patients by phone, email and webcam. They’re also consulting with each other electronically—sometimes to make split-second decisions on heart attacks and strokes. Patients, meanwhile, are using new devices to relay their blood pressure, heart rate and other vital signs to their doctors so they can manage chronic conditions at home.”

It’s not just locally that this makes sense, either.

“Telemedicine also allows for better care in places where medical expertise is hard to come by. Five to 10 times a day, Doctors Without Borders relays questions about tough cases from its physicians in Niger, South Sudan and elsewhere to its network of 280 experts around the world, and back again via the internet.”

While it might seem a new idea, “More than 15 million Americans received some kind of medical care remotely last year, according to the American Telemedicine Association, a trade group, which expects those numbers to grow by 30% this year.”

But not all is bright on the horizon. There are serious questions about the quality and qualifications of the doctors who practice telemedicine, and questions about who will foot the cost. “Some critics … question whether the quality of care is keeping up with the rapid expansion of telemedicine. And there’s the question of what services physicians should be paid for: Insurance coverage varies from health plan to health plan, and a big federal plans covers only a narrow range of services.”

“Critics worry that such services may be sacrificing quality for convenience. Consulting a random doctor patients will never meet, they say, further fragments the health-care system, and even minor issues such as upper respiratory infections can’t be thoroughly evaluated by a doctor who can’t listen to your heart, culture your throat or feel your swollen glands.”

As to the question of qualifications, the WSJ discusses one example:

“First Opinion, connects users with doctors in India for web chats, but a disclaimer states that these are merely ‘social interactions.’ If a prescription or lab test is warranted, a locally licensed doctor joins the conversation for a $39 fee. The company didn’t respond to requests for comment.

“Are such services ‘practicing medicine’ without a license? The exact definition varies from state to state, and state medical boards generally don’t investigate unless a patient files a formal complaint. Even then, boards have jurisdiction only over individual doctors licensed in their state, not companies, or physicians overseas, says Ms. Robbin of the Federation of State Medical Boards.”

There’s also the problem of “branding” medical providers (though we have that already, with mega-medical groups). “Telemedicine is also shaking up traditional relationships between providers and payers and fueling the rise of medical ‘megabrands’ whose experts are increasingly competing for patients in each other’s backyards.”

It will be interesting to see where all of this leads. People are not happy with the current state of insurance and medical care available in the US, and elsewhere in the world. What Brave New World awaits us on this front?

Wrangling Atoms to Create Rewritable Memory

WEB_GettyImages-153665968
Shelves of paper catalogues in the main reading room of the US Library of Congress Thomas Jefferson building. (Alex Wong/Getty Images)

I saw this article on nature.com and paused. I wouldn’t even have glanced at the article a few months ago, but that was before I began the ongoing edit of a book on parallel process coding to increase the speed of applications (I know, it rolls off the tongue, doesn’t it?).

But here was a new angle, using atoms to overcome the physical limitation of Moore’s Law to increase memory in smartphones and computers. Moore’s law famously predicts that the number of transistors people can squeeze onto memory chips will double every couple of years, but as of now, Moore’s law seems to have reached a limit: technology cannot be miniaturized indefinitely.

Researchers are trying to get around this limitation by starting small—using individual atoms—to make big gains in data-storage capacity. Now, a team has developed a 1-kilobyte rewritable data-storage device using chlorine atoms arranged on a small metal surface. According to a researchers report on 18 July in Nature Nanotechnology, if the team expanded that surface to one square centimeter, it could hold about 10 terabytes of information. This is HUGE news!

“It’s by far the largest assembly on an atomic scale that’s ever been created, and it outperforms state-of-the-art hard disk drives by orders of magnitude in data capacity,” says lead study author Sander Otte, a physicist at Delft University of Technology in the Netherlands.

Otte and his team have arranged chlorine atoms into square grids on a copper surface, and then placed those grids side-by-side, like uninterrupted terraces. Each grid contains a few empty slots, or holes. This allows the research team to move atoms around, much like sliding pieces around in a tile puzzle. Each line on a grid encodes one unit of digital information called a ‘byte.’

Otte’s team uses a scanning tunnelling microscope with a sharp needle, like the tiniest of tweezers, to probe the atoms and make them hop into adjacent spaces. One chlorine atom and one vacancy make one bit (there are 8 bits in one byte). Moving chlorine atoms in and out of vacant spots means researchers can switch between ones and zeroes, the basis for all computer code.

nature-nano-resized
This pattern of atoms and vacant spaces encodes the letter ‘e’ (Courtesy TU Delft)

One of the big drawbacks of this device is that it must be kept at –196 °C: the boiling point of liquid nitrogen. This is a far cry from room temperature. But, “It’s very nice proof-of-principle work, demonstrating the first step of applying this technique of atomic manipulation to something that could lead to a functional memory device,” says Stefan Fölsch, a materials physicist at the Paul Drude Institute for Solid State Electronics in Berlin.

If researchers can find a solution or work-around for the current physical constraints and limitations to this atomic technology, they could eventually scale the technology up to larger structures and arrange their grids in three dimensions, then one could pack hundreds of terabytes—equivalent to all the information contained in the US Library of Congress—into a cube the size of a grain of salt. Further improvements could prove useful to data storage in the cloud, reducing the need for new data centers, which are currently dotting landscapes around the globe like fast-growing lichen.

But data storage is just one application. “Otte’s research gets people interested in thinking about what we want to do on an atomic scale,” says Chris Lutz, a staff scientist at IBM Research at Almaden Research Center in San Jose, California. In the long term, Otte and his colleagues’ research could pave the way to designing new materials, atom by atom.

A huge fan of nanotechnology, I hope to be around for new developments in the years to come. I KNEW that small could be dynamic!

 

College Kids’ Dream: Self-Cleaning Clothes

sunlightcoating

Welcome the upcoming advent of the college kid’s dream: self-cleaning clothing, as reported in Gizmag and elsewhere.

I love drying clothes outside, but it’s getting harder to do in big cities, or in condo environments where such a display of “low status” is frowned up. But drying clothes outside is healthier for you than drying them in a machine inside, due to the risk of generating harmful dryer emissions inside.

Plus, sunlight kills germs, so not only do your clothes have that fresh-air scent, but the little beasties that might still be hiding in the cloth are likely now defunct.

Thanks to research by scientists in China, we might soon not only revert to drying our clothes outside, but we might also end up cleaning our clothes by hanging them outside.

Scientists in China have reportedly successfully removed orange dye stains from cotton fabric, through the power of sunlight and a special fabric coating.

Mingce Long and Deyong Wu created the coating, which combines titanium dioxide (a good sunblock agent) and nitrogen (this element forms about 78% of Earth’s atmosphere and is the most abundant uncombined element).

When exposed to sunlight, dirt on fabric treated with the coating breaks down, and microbes die. While the coating in its basic form is effective, it was found that it does an even better job at dispersing dye coloration when silver and iodine nanoparticles are added. Additionally, it is able to remain intact and active after washing and drying.

Although light-activated self-cleaning fabrics have been created previously, all of those required concentrated ultraviolet light, as opposed to natural sunlight. Good news for consumers, who have more access to sunlight than to concentrated ultraviolet light (unless they are growers of a particular consumable).

Should the dirt-dispersing coating eventually be commercialized, however, there are doubtless many people who would want nothing to do with it. Although titanium dioxide is now an active ingredient in products such as sunscreen, cosmetics, and paint, studies have shown that it can cause genetic damage in mice and brain damage in fish. This has led to concerns over the effect that it could have on humans, and the environment.

Remember fire-retardant pajamas for little kids? Researchers found the chemicals coating the fabric in the pajamas were to blame for:

  • thyroid disruption
  • early onset of puberty
  • cognitive problems
  • delayed mental and physical development

The chemicals used to make pajamas and other consumer products flame-retardant also showed up in water, wildlife, and human breast milk.

So, perhaps we’ll tread lightly into the sunwashable fabric promise, lest we look great but cannot remember our names.

A paper on the coating was recently published in the journal ACS Applied Materials & Interfaces.

I’m Rubber, You’re Glue: Icephobic Coating Repels Ice

ice

Generally-speaking, hard objects and soft objects don’t bond well together. It comes down to a principle known as ‘interfacial cavitation,’ in which one surface (the soft one) deforms under pressure, while the other (the hard one) does not -– as a result, they pop apart. Now, scientists at the University of Michigan have used that principle to develop one of the most ice-repellent coatings ever made. Its applications could range from airplane wings to car windshields to freezers. (Those are the logical applications; I imagine all kinds of unique applications will follow, as people use their imagination…)

Previously, most anti-ice coatings have been rigid and slippery, with the idea being that what repels liquid water should also repel ice. According to the U Michigan team, however, that line of thought was flawed.

“Nobody had explored the idea that rubberiness can reduce ice adhesion,” says associate professor of Materials Science and Engineering Anish Tuteja. “Ice is frozen water, so people assumed that ice-repelling surfaces had to also repel water. That was very limiting.”

Instead of creating an ice-repelling solution, the team, led by Tuteja, created a rubbery coating, made from a blend of commonly available synthetic rubbers, such as polyurethane. The resulting clear spray-on coating, while somewhat tacky to the touch, might at first seem like it would “hold onto” the ice, but the fact is that the hard ice easily releases when the soft rubber deforms -– and the hardened ice slips off the surface. All that is required is the force of gravity, or a slight breeze.

By tweaking the composition of the coating, it is possible to select for factors such as durability versus ice repellency. This means that a coating designed for airplane wings could be very durable, as high winds would already blow much of the ice away, while a coating for industrial freezers would be more repellent, allowing ice to be shed with little effort.

icephobic-coating.jpg

The coating would also find instant users in the frozen North, where keeping windshields ice free is a constant battle. Despite not being able to repel water, the rubbery coating was able to stop ice from setting on the surfaces because of the previously mentioned interfacial cavitation. As the rubbery surface changes forms when exposed to even small amounts of force, it effectively prevents ice from becoming tightly bonded to it.

Put Your Voice in a Pot; Pour It Out When Done

Remember those cartoons you watched as a child, where people would be speaking in a snowstorm or on a bitter cold day, and their voices would freeze, and they’d have to take them home to thaw out the words out to hear what they’d said?

Well, today, “freezing” your voice for later has a new twist. It’s not a tape recorder, but something more fun: it’s a teapot recorder. Called ‘Otopot,’ the device lets you store and then pour away your voice like water.

otopot-1

It’s a simple, and captivating, idea.

To begin a recording, you simply remove the lid on the pot. You can then record up to a minute-long message.  You could serve the rechargeable pot alongside your significant other’s cup of coffee in the morning, or place one on a child’s nightstand after he or she falls asleep at night so they have an uplifting message to greet them in the morning. Imagine your voice flowing like water…

Once the recording is heard, the recipient can listen to it repeatedly simply by removing the lid. When it’s time to erase the message, one simply tips the pot as if pouring out water and the recording vanishes. The pot can only hold one recording at a time, so it becomes a fun way to hold a very unusual form of conversation.

Sure, it’s a simple matter to leave a sticky-note message for a loved one or family member, but how much more appealing would a voice memo be, complete with verbally conveyed emotions?

Click here to see how to use the James Dyson Award-winning Otopot, and the components of the talking water jug.

otopot

Understanding that people might like to send clever messages in something other than a pot, the designers created a removable ‘Otopot core,’ which can be placed inside a birthday box or any other object with a lid.

“OTOPOT” was nominated for a grand prix and awarded 2 sponsor prizes at GUGEN 2014 (a largest competition of original electrical hardware in Japan).

Currently, the student inventors, Takeshi Katayama and Shota Kumiji, are finishing up their studies but say they plan to launch the concept via a crowdfunding campaign later this year.

Will it change the world? Likely not. Will it captivate the imagination of creative types and lovers? I would think so. It would also lead to misuse, I would think, to send mean messages instead of loving memos. In that case, I would suggest the name StinkPot.

3-D Printed Ear Bones Grow in Mice

bioprint-living-ear-11

I am neither an entrepreneur nor a technology visionary. (I had initially scoffed that Web pages would simply be glorified Yellow Page ads.) I am not an early adopter of technology. I am more a third-generation user, typically all-in once my skepticism has been allayed. (While I was content with my flip-phone with its rudimentary camera, I am THRILLED that I discovered the elegance and power of the smartphone. How did we ever live without an iPhone?)

Needless to say, I scoffed at the idea of 3D printing. So you can print 3D puzzles, big deal. So you can print a model tower for an architecture presentation, big whoop. Oh, me. I am so NOT a visionary!

Today’s blog is on how 3D printing is changing science and medicine, one tiny ear bone at a time.

According to Gizmag.com, 3D printed tissues and organs have shown real potential in addressing shortages of available donor tissue for people in need of transplants, but the biggest obstacle to success has been having them take root and survive after implantation.

But in an epic life-changing success, researchers at Wake Forest Baptist Medical Center have used with a newly-developed 3D printer to produce human-scale muscle structures that matured into functional tissue after being implanted into animals.

Researchers have been exploring bioprinting as a means of replacing damaged tissue for several years now. The difficulty in replicating the complexities of human tissue has been extremely difficult, however, with scientists testing the waters with specialized bio-inks and various purpose-built printers in an effort to produce usable, engineered tissue. Every failure is a success, though, in the search for the perfect “ink” for such enterprises.

After more than a decade, researchers at Wake Forest Baptist Medical Center are engineering structures of adequate size and strength to implant in the human body, using the team’s Integrated Tissue and Organ Printing System (ITOP), which is claimed to overcome the limitations of previous bioprinting approaches.

It spouts water-based gels that contain the cells, along with biodegradable polymers arranged in a latticed pattern and a temporary outer structure.

bioprint-living-ear-4

The water-based gels were optimized to promote cell growth and health. This, combined with micro-channels that allow nutrients and oxygen from the body to permeate the structure, allows the system to remain alive while it develops a system of blood vessels.

To demonstrate its capabilities when it comes to soft tissue structures, the team used the system to produce muscle tissue, implanting it in rats and finding that two weeks later it was robust enough to permit blood flow and induce nerve formation. Using human stem cells, the system also printed jaw bone fragment large enough for a facial reconstruction and implanted them in rats. Five months later, the structures had matured into vascularized bone tissue.

bioprint-living-ear-12

Previously, engineered tissue structures without ready-made blood cells needed to be smaller than 200 microns in order for the cells to survive, but this new approach solves that problem. The researchers used ITOP to produce baby-sized ear structures measuring 1.5 in (3.8 cm) long, which were implanted under the skin of mice in the lab and went on to show signs of vascularization one and two months later.

Further adding to ITOP’s potential is its ability to take data from CT or MRI scans and make individually designed (bespoke) tissue for patients. So if a patients is missing a particular piece of tissue, such as a section of ear or nose, for example, the system could theoretically reproduce a precise replica.

“This novel tissue and organ printer is an important advance in our quest to make replacement tissue for patients,” says Anthony Atala, senior author on the study. “It can fabricate stable, human-scale tissue of any shape. With further development, this technology could potentially be used to print living tissue and organ structures for surgical implantation.”

The researchers will continue to explore the approach to track longer term results. Their current study is published in the journal Nature Biotechnology.

Technology limited only by imagination. H.G. Wells and Mary Shelley, what do you think? Any ideas? And Edward Scissorhands, ready for a makeover?