Let's stop innovating failure

So today I’d like to posit some overall thoughts on the “Internet of Things”, specifically pointed at the innovation space and getting products to market. This is quite outside of my company’s normal wheelhouse, but I think our hands on perspective of what hits the streets (and our desks) will be unique enough to pique interest from a broader audience. For those in the security space, I guess I should apologize for the lack of direct exploitation techniques, but this diatribe is really meant for the broader group outside of our walls.

So, you (or your company) have a great idea to change the world and make a huge profit. First of all, congrats and good luck. All you have left to do is write some code, design some hardware and package it all together into an amazing widget. The only problem you face is a temporal one… if you thought of something, chances are others are also thinking about your problem, and hence the race to market begins. 

Speaking of your market, my view is that it functions in 1 of 3 distinct ways:

• You find yourself entering the market as a sole innovating product with no direct competitors
• You have a unique and special flair for your product, but you are not alone in the space.
• You are a big name vendor that is entering the space. 

Realistically, the “big name vendor” should not be a separate bullet. Your specific product falls into either of the “novel” or “iterative” spaces listed, but your name and power come with some additional considerations. The new device you are shipping will benefit from your company name and the reputation you have with customers, but if the product fails you will see negative impacts outside of your space. Simply, if your newly enhanced smart TV ships with copious numbers of security flaws, that branding might negatively impact other business units.

For the “iterative” space, you will benefit from the general advertising across your competitive landscape. This effect should be considered cumulative, if not exponential, as the barrage of adverts self validates the product as needed and required from the consumer view. Given you don’t have to sell the overall vision, your advertising tends to focus not on the overall technology but on your special sauce.

For the “novel & new” product designs, you are laden with selling the concept that the market has a hole and consumers simply can’t live in a world with such a void. As your product is alone in it’s space, it sells itself once the consumer is convinced of the overall void.

Now, I mention the advertising space because I believe it embodies a considerable amount of how you as an innovator personally see your product. It is a faint echo of what your priorities are and a telling exhibit of how you are specifically positioned in the market. It also allows me (as a security researcher) to assume what level of security to expect in the product. I do this by extrapolating your security design as a function of “assumed priorities” and “time from design to market”, generally the faster you race the weaker I expect your implementations are. It is logical (although a bit cheating), but the metric seems valid from analysis to date.

In general, no company is going to reinvent the wheel. There is no need, it costs too much and is generally a bad idea (hopefully gone are the days of rolling your own crypto). Instead, I predict your product will be based on a fairly standard design:

• You have some embedded processor running Linux (or if you hate me personally, Windows)
• If you are constrained or concerned about speed / size / power, you drop linux and ship with an RTOS or code on bare metal instead
• Following good software engineering principles, your code is modular and uses large swaths of code not written specifically for this project (libc, WiFi and IP stacks, display drivers, etc.)
• You build your product around a known processor and follow design guidelines from the manufacturer, possibly even basing your design on a template provided by the vendor
• Knowing mistakes happen, you ship your device with debugging capabilities enabled or easily re-enabled… this helps with RMA problems and figuring out how fielded devices fail

This in essence is efficient design, you offload as much engineering as you can on previously proven and fielded devices and leave only the hard and specific problems to yourself. You cut costs and gain a robust support architecture. Aside from the last bullet, this mentality it is a sound. But in reality nothing is free, and these choices have the following consequence:

As a researcher, I am already familiar with your product before you ship it

• I know the system on chip reference designs and suggested hardware designs
• I know the operating system particulars
• I know how to debug your product
• I am aware of the known issues in your building blocks

In general, this is why I see innovation “Time to Market” as a realistic metric for “Time to Failure”. Your wonderful engineering practices and time saving approaches leave your product prone to exploitation. In a grim reality, your largest saving grace when racing to push things to market is the actual market saturation itself; the case is not “is your device insecure” as much as “has anyone spent the time to look at it instead of a competitor or another device”. I honestly don’t mean to be dismissive here, I respect the hard engineering problems you have overcome. I’m simply stating that I find it rare that a “novel new product” is actually new or unknown to us in the security industry. To borrow from the above example, your new smart TV is identical to the embedded industrial control system I looked at last week.

I promise we are almost past the dour news, but simply stated the following assumptions appear to drive your "idea to design to development to shipping" cycle:

• Assumption #1: Your special sauce is cutting / bleeding edge and highly innovative. You race these concepts to market before they are secure expecting protection from your building blocks. This is highly understandable from a market perspective, though maybe not the best idea.
• Assumption #2: You assume your selected platforms and building blocks are secure because they are fielded by others. This is sadly incorrect in a general sense. The vendors in that space are following assumption #1 and you are simply the powerless consumer.

We all want to stand on the shoulders of those that came before us, but we need to take the time to understand the limitations of those principles.

Your product will never be 100% secure, nothing is. That is no excuse to not understand your actual security posture and the inherited posture of your base platforms. It is honestly a hard problem, so I will suggest a mindset that might help:

Aim for is a level of security based on the ROI of what you are protecting. 

Understand the financial impetus of your market and your potential attackers. Respect your consumer and their needs as much as you respect your own device. This is a simple idea that should be obvious, but I see it disregarded too often to not specifically mention.

It is not that linux and your other “off the shelf” building blocks cannot be configured to be secure enough for your use case, the real problem is that in your rush to market you assume they already are and nothing could be further from ground truth.

What you can do:

• Stop assuming security is someone else’s problem
• Start considering what potential your product has to be misused
• Don’t just test your product with a “happy test path” mentality
• Actively ask your engineers to break your design BEFORE you commit a single line of code or anything to silicon and don't write code until they fail at this task
• Continue this process until you ship your device
• Continue this process as you iterate designs on a shipped product
• Ask for help
• Consider what side effect implications your product has to your customer environments

I don’t mean to shill our services here, but seriously you should ask for external eyes on your design and your product from someone qualified to help. Don’t think that just because we are security consultants or hackers that we are the enemy. Semantics aside, we are honestly here to help. Hire someone to tell you your weak points and take heed to the reports. Consider it an integral part of your marketing budget; I promise it will be considerably cheaper that trying to play publicity cleanup once issues are out in the open. 

If you want a product with longevity and a customer base that will continue to buy into your brand and vision, you need to respect them. Take care of them, protect them and don't assume someone else will. The security of your customer is just as much your responsibility as delivering a new device to them is. The decision may delay your time to market by a week or a month, but the end result will be a better product and a better world. 

Making a 1st Impression with Hardware

Fluffy Introduction Text

I get the “What exactly do you do for a living?” question fairly often, and tend to generically toss out a canned response of “hacker” or “information security professional” or “computers” depending on the person asking and whether or not I’m being questioned by a customs official with a gun and a surly look. I thought it might be fun to blog up a example or two of what we actually do for a paycheck, as it might help explain our niche inside a niche industry and maybe illuminate where our customer’s money goes and what value we provide.

For this exploration, let us focus on 2 random devices sitting on my desk this weekend: The Samsung Galaxy Note 3 (specifically the SM-9005 variant) and the often misunderstood and slightly obscure Samsung Anyway S102.

For the Note 3, I’ll detail what a teardown looks like to me and why it matters to the customer. The device is fairly well documented on the internet, so a full and complete hardware reverse engineering analysis probably isn't worth the time. As such, I’ll just focus on the high level overview. Consider this my first couple of hours on a project, simply what I look for and what my plan of attack would be. This teardown will disregard software and the USB stack entirely, and just be a glance at the hardware.

After the SM-9005, I’ll walk through a deeper analysis of the S102. I expect this to be quite interesting and fun, though it will be a separate post that will go up within the next couple of weeks.

So, without further ado….

The Samsung Galaxy Note 3 (SM-9005) Teardown

With any common mobile platform that crosses my desk for analysis, I start with a quick perusal of iFixIt for a teardown. For the Note III, we find this: <iFixIt Link>

The device is fairly straight forward to crack open, but the public teardown doesn’t tell us much about the internals. We can fix this with a bit more time on google and quickly run into the beloved chipworks.com tear down (we will come back to this later): <ChipWorks Link>

While the overall information is nice, what we really care about here is:

• What components are being used (ChipWorks gave us a listing already)
• How was the device built
• Did the designers leave any debugging mechanisms exposed or active during production
• Are there any weak or squishy portions of the design that look easily exploitable

So with the background we have at hand, it’s time to take a screwdriver to the phone and see what’s inside for ourselves.

A GENERIC TEARDOWN PHOTO

Taking a closer look at the main PCB, we see all the same components and latch plugs that the other tear-downs exposed. We also get a first pass understanding of the device complexity and it’s general craftsmanship, and in this case the Samsung engineers design stays true to form with a clean and well structured PCB. All the components are well placed and the overall view is aesthetically uncluttered. We can quickly see the camera in the center, as well as the SIM card slot and the battery connector. The large chip below and to the right of the camera is the main slab of NAND Flash on the device. The other components we will revisit after we finish our “lay of the land” exploration.

TOP OF THE PCB

Further analysis of the phone shows the case connections are just as cleanly laid out. Typically there is little of interest on the case and screen sides of a phone, but on the Note 3 we can see a handful of interaction points that drive the screen and touch sensors underneath the PCB. The small silicon on the right (below the headphone jack) is the Synaptics S5050A touch screen controller. If the interest of this project was to manipulate user interaction points, this would be our starting place. Aside from the controller and the pads on the left, there is little on the screen side but dense connection cables, and these tend to be easier to explore on the main PCB.

SCREEN SIDE OF THE CASE, UNDER PCB

Our last view of the entire PCB will be from the bottom. This is where our main SoC (system on a chip) is hiding, along with a collection of other interesting components.

Starting with the top left corner, we can see where the PCB contacts the pressure connection pads on the case that I mentioned before. I this were a real project I would eventually explore them with a multimeter to be sure, but my initial assumption is that these simply share power and ground with the screen.

Moving to the top, we can see the 4 pins that connect to the battery when the phone is assembled, and to the right of that is our main slab of RAM followed by 2 sets of RF Shielded components.

BOTTOM OF THE MAIN PCB

If you didn’t notice, I did a bit of “it’s magic” handwaving when locating the SoC (it’s hiding underneath the slab of RAM). We know this because we googled the device and ChipWorks already discovered this for us and they tell us it is the new Snapdragon 800 (MSM8974). Had this not ben the case, I would have to start desoldering components looking for Qualcomm silicon. While cleaning a PCB of all components gives me great joy, this phone was a present and I’d like to at least boot it once before we kill it, so we will take the ChipWorks finding as gospel for the time being.

So far, our analysis is far from special or unique. iFixIt delves to this level trying to help people fix or replace broken components on a phone and ChipWorks delves a bit deeper looking for new hardware and trends in the engineering space. At this level, we also diverge, but our path is to catalogue hardware interaction points and to explore the PCB for potential debug interfaces.

Jumping down the stack a bit

Now, before we delve much deeper, it’s probably a good idea to spin up a quick vernacular, especially if you are not experienced in hardware. First, modern PCBs are typically dense and multi-layered. As such, a simple visual inspection is not always going to grant you 100% understanding as we can’t see internal sandwiched layers from the top or bottom. Phones and high end consumer electronics are especially dense, while more esoteric hardware still tends to be dual layer (top and bottom only).

When wanting to connect a trace between layers, a via is used. These are simply the “dots” you see from visual inspection and they generally mean that a trace has jumped to another layer (though they can be use for other things as well, like pulling a reference voltage or ground signal from a different layer).

Lastly, when PCBs are being mass produced they need to be tested prior to deployment. While a multitude of options are available, the majority of devices we look at in the real world still support the concept of a “bed of nails”. <insert google image search results here>. This concept basically means that interesting or problematic areas of the board have testing points exposed on the outer layers of the PCB so that an external testing machine can quickly validate them on the production line. Furthermore, my favorite hardware designers tend to expose debug functionality with these interaction points to allow for firmware and OS flashing post production.

Now that we are loaded with a simplified lexicon, we can delve a bit deeper into some closeup photos of the SM-9005 board.

The first thing I notice upon glancing at the PCB is the bed of nails pads lying about the board (these are the copper / gold circles in the below picture). If this wasn’t a Samsung device and I had not spent countless days tracing prior hardware revisions of these pads, I would be really excited. I’d immediately start tracing the pads with a multimeter and tapping them with a logic analyzer to explore what they are exposing. Given the target though, we will wait on that task until the S102A analysis (where I expect much more interesting results).

BED OF NAILS PADS AND CABLE SEAT

The other thing to take note of in this picture is the wonderfully large spacing of leads off the ribbon cable seat. These pinouts, while still fairly tight and small, are spaced far enough apart that latching with a logic analyzer to watch all the data pass over the cable is an approachable task and much easier than tapping the cables directly. If we cared about PCB to Screen interactions, this would be our starting point.

While we will disregard the bed of nails pads because I’ve explored Samsung phones in the past and know better, I do see something on the board that makes me second guess that  decision. Below, you will see a collection of unpopulated pads that are designed to seat standard capacitors and resistors. Seeing these gets me quite excited, especially because some of the traces end in nail pads as well.

You will rarely see PCB layouts with truly unused components, everything about physical hardware is about tradeoffs between space, cost, usability and mass production. Engineers don't simply design and produce extra unused pads as a rule, so if they are on production equipment they probably have a function.

UNPOPULATED PADS

In reality, these are probably laid out to support the different hardware variants that Samsung utilizes around the world. In this manner, the designers can lay out a single PCB and mass produce it instead of full redesigns for each variant of the hardware that is to ship. The factory would then populate the board with different component sets depending on whether or not the phone was to be an AT&T LTE device or something else.

Aside from the logical, these pads could expose something immensely more interesting: hardware debugging and firmware manipulation. At some point in production, the device must be loaded with software and most common SoC vendors protect that functionality with a series of hardware flags controlled by resident voltages. In this case, I would start tracing the pads with a multimeter to see where the missing discrete components would effect, and what circuit they could complete if bridged.

We have seen hardware of a lower mass production scale where through hole resistors were utilized to open the SoC for firmware debugging and then simply clipped by hand before the final device closure and shipment to customers. If my target was to acquire Qualcomm firmware or debug the resident NAND, these pads would be my starting point. Further enticing me along this path is the mysterious pads exposed directly next to the SoC / RAM stack (as seen below).

PADS ALONG THE MAIN SOC / RAM

Final Thoughts

While we’ve meandered around a bit without real focus, I do hope I’ve explained a bit about what I look for in a teardown and what excites me to see. If this were a real engagement to find security related issues on the Note 3, I would be tracing all the pins and attempting to recreate the full PCB schematic. I would then acquire spec sheets for every piece of silicon I could and start looking for debug or flashing capabilities. The main focus of this part of a hardware analysis is to plan an attack to grab the resident firmware and the Note 3 exposes a couple of paths that while probably dead ends are none the less worth running down.

We would also explore the USB side of this hardware and obviously look at the driver/os/kernel as well, but I think I’ll save those walkthroughs for an easier device and another time / blog post.

Thanks for reading,

m0nk

if you have any questions, please email us or find us on twitter @m0nk_dot / @Atredis