For Engineers

Introduction#

001. "Security Training for Engineers, April 2018"

Welcome to "Security Training for Engineers". This talk is specifically geared towards engineers, and is going to be a bit more technical than the last training. If you're not an engineer though, don't worry, you're more than welcome to stay. While the topics will be technical in nature, it should be informative no matter what your skill level is, whether you're just getting started, or are a seasoned engineer with years of experience.

For those who've not met me yet, "Hi, I'm Rich". I work on security and incident response, and make lots of silly jokes. You'll get used to it.

Anyway, let's get stuck in.

Testimonial#

002. Testimonial quote.

If you've joined PagerDuty in the last year, then this will be your first time going through this particular training course. It's had rave reviews in the past, as you can see here from this completely genuine testimonial quote, which I totally didn't make up in any way.

Classification#

003. Classification ratings for these slides.

Before we delve into the main content, I wanted to give you all a quick reminder of our slide classifications. We went through this at the last training, but the same applies to this one too. Please respect the classifications that are added to these slides and don't share ones which shouldn't be shared. As with our previous training, this one will also be open-sourced so you can share it with friends and family, and the internal version will be available on the wiki along with the recording.

Goal#

004. Goal of the training.

So why are you all here today? The goal of today's training is to give you the opportunity to learn about a variety of common security vulnerabilities that are relevant when developing software. I'm not just going to show you what they are, and how to stop them, but I'm also going to show you how to exploit them too. Exploiting these vulnerabilities is usually the best way to really learn how they work, and is an important part of knowing how to stop them. Plus it's more fun to show you how to exploit them.

Topics#

005. List of topics we'll be covering.

I’m going to be covering a lot of different topics. Don't be put off by the quantity here, some I’ll go into quite a bit of depth on, but others I’ll just be touching on briefly. Some of these may already be familiar to you, but hopefully you’ll still learn something new if that's the case.

Warning#

006. Warning sign.

But I want to kick things off with some quick warnings...

Framework#

007. "The framework takes care of that for me..."

I often hear this statement from developers, that their framework of choice takes care of all the nitty gritty security things for them, so they don't need to understand how they work. I'm here to tell you that that is completely not the case. Rails, Akka, Symfony, whatever you use, you can't just blindly assume your framework or library of choice has you covered. Yes, it's convenient that they do all the heavy lifting for you, but you can't just abdicate your responsibility to understand and be able to repair security issues yourself.

Framework Issues#

008. Rails vulnerabilities. Reference

Because even though frameworks are great, they are not immune to security issues (Spoiler: No software is immune to security issues). If a new vulnerability is publicly announced, you can't always just sit around and wait for an official patch. You will need the knowledge of how the security vulnerability works so that you can patch it yourself. Because attackers aren't going to wait around, they're going to start exploiting them immediately.

Now, I'm certainly not advocating against using frameworks, you definitely should, I'm merely saying that you should never assume the framework will always protect you.

Don't Trust Blindly#

009. Don't trust frameworks blindly.

Basically, you shouldn't blindly trust frameworks. It’s important to understand the underlying principles, so you can tell when the framework isn’t doing what it should, and so you can quickly fix things when it turns out they’re not.

Hackday#

010. It's just for a Hackday.

Another statement I hear occasionally is "It's just temporary for a Hackday", often used as an excuse for attempting to ignore security best practices.

Hackday Security#

011. Hackday security.

Just because it's a Hackday, it doesn't mean you can ignore the rules. I know you want to make something work quickly, and you don't want to mess about with all this "security" stuff. But it's important and should not be ignored. You shouldn't be trying to disable firewall rules for Hackdays (and yes, we do get immediately paged if you try), and you shouldn't use a public code repository to develop your Hackday idea, it's fine to open-source it later, but sometimes you can give away information we don't want to be public yet.

Finally, you should never ever be using customer data for your Hackdays, unless you're at the stage where it's getting pushed to production. We went over this on our last training, but I'll reiterate it again, customer data should never, under any circumstances, leave our production environment.

Story Time#

012. Story Time.

With that out of the way, let's jump into a bit of story time.

Critical Bug#

013-020. Redacted slides.

Every System Has Security Issues#

021. Every system has security issues.

The point I'm trying to make here is that we're not perfect. Every system in the world has security issues. An external researcher with no access to our source code (we hope) was able to find this bug. It's up to all of us to be on the lookout for things like this ourselves too.

It's not all doom and gloom though, there was a happy ending here...

Happily Every After#

022. Redacted slide.

SQL Injection#

023. SQL Injection.

OK, let's get started with some actual technical content. Our first main topic is SQL injection.

SQL Injection Definition#

024. SQL injection definition.

What actually is it? This is the simplest one-liner I could come up with. It's any user input which is being executed in an SQL query at runtime. That means the user is able to manipulate a database query based on their input.

Login Page (1)#

025. Login page SQL.

Let's take a look at a quick example, using the query for a hypothetical (and very bad) login page. Seriously, never design a login page like this, I'm just using this as a contrived example to demonstrate.

When logging in, this will lookup the user record based on the username and password that is entered. If a result is returned, it will log that person in. Pretty simple, and there are actually login pages out there that work this way (unfortunately).

Login Page (2)#

026. Login page working example.

So let's login with a real user to see how this works under normal conditions. The username and password get put into the query, which will then execute using those values.

Login Page (3)#

027. Login page results.

The SQL statement is valid. The username and password are correct, so a single result is returned for the user, and they would now be logged in as this user.

OK, so that's all well and good. That's how the developer intended it to work.

Login Page (4)#

028. How an attacker uses it.

But honest people aren't always going to be the ones using such a login form. Some users will be attackers, and they won't play by the rules. So let's say we have an attacker, and they now enter something like this instead. They have a username of admin, but a password of ' OR 1=1 --.

As before, the username and password get replaced in the SQL statement. But something different has happened here. Because this code is susceptible to SQL injection, the query being executed has actually changed from what the developer wanted. You can see in the red box here that the SQL query is now different.

Login Page (5)#

029. SQL has changed.

This new SQL statement has an extra condition in it, which means this query will return the row for the admin user, even though the correct password wasn't entered. The query added an OR 1=1, which will always evaluate to true (since 1 is always equal to 1). Since OR true will always be true, it doesn't matter what else the user entered, they will automatically get the correct result.

This user would now be logged in as an administrator even though they didn't know the password. That's not great.

Login Page (6)#

030. Dropping a table from a login page.

This can get out of hand very quickly. Hopefully you can start to imagine other ways in which this could be dangerous if your code is susceptible to SQL injection.

Bobby Tables#

031. Little Bobby Tables. Reference

You may have come across this XKCD comic before, about a boy called Little Bobby Tables. If you didn't know what SQL injection was before, you should hopefully now start to have an idea of what's going on here.

Companies House#

032. UK SQL injection company. Reference

There’s even a (completely legitimate) company in the UK with an SQL injection as their name. The Companies House site isn't susceptible to SQL injection, so this all works perfectly fine.

Users Shouldn't Modify SQL#

033. Users shouldn't modify SQL.

The key takeaway here is that you should never let your users directly modify the SQL being executed. Users should be providing values only, and not be allowed to alter the structure of your query. If you need to have different queries executed based on some value the user picks, then they should be two distinct queries in your code.

Lookup by ID#

034. Lookup by ID.

Now that we've seen a ridiculous and contrived example, let's look at something a little more realistic. Here's a typical query you might see which looks up a person's name based on their ID. This is a pretty standard pattern you'll see in a lot of codebases.

Typical Usage#

035. How it's intended to be used.

Under normal conditions, you're meant to provide an ID number, and you'll get the user's name in return. This is how it's intended to be used. But if it's susceptible to SQL injection, there's a lot more information you can get out...

Wildcard#

036. Wildcard.

What if we use a wildcard % instead of an ID number? Now we'll be able to see everyone's name from your user table. Something like this could allow an attacker to steal a full list of your users. But it's not too bad right? They can only get names and nothing super important like passwords...

Union#

037. Union.

Unfortunately, once you have a single instance of SQL injection, it's pretty much game over and you can grab almost any data that you want. Here we can use a UNION to steal the usernames and passwords for every user too. But it's not too bad right? You can't access files on the filesystem or anything...

Load File#

038. Load file.

Like I said, it's pretty much game over once you have an SQL injection point. You could use LOAD_FILE to load things from the filesystem too.

Blind Injection#

039. Blind Injection.

So far this example has required the output to be presented directly to the user in order to be useful. But even if that's not the case, you can still extract information, it just takes a bit longer. There's a technique called "Blind Injection" which can accomplish this. There are two main types...

Boolean#

040. Boolean.

The first is boolean, where you can tell the database to throw an error depending on if a certain value is true or false. It'll take a while, but can easily be scripted to extract data.

Time-Based#

041. Time-Based.

The other is time-based, where you can use the same technique, but add a time-delay and use that to extract the true or false value. Once you have an SQL injection, attackers can be very sneaky in the way data is exfiltrated.

Escaping?#

042. Escaping?

So how can we start to protect against SQL injection? Some folks like to try and add checks that will stop a statement from executing if it has "bad" words like DROP in it. Unfortunately, that isn't going to work. Not only would you need to maintain such a blacklist, and be careful to make it case-insensitive, etc. But there's always going to be ways around it.

Likewise, you might think that you can just escape all quotes from user input in order to stop the query being modified. Depending on the language you're using, these types of checks can also be bypassed in various sneaky ways. There's always going to be some strange Unicode character you hadn't thought of that gets interpreted as a quote, or something silly like that. It's not really worth investing in these types of checks.

Parameter Validation?#

043. Parameter Validation?

What about parameter validation instead? If we only expect a field to contain an integer, we can just reject any input that's not what we expect it to be. This would certainly work for integer fields.

If a field only expects alphanumeric characters, again you can just restrict to those characters. This would also work, but you're going to start running into issues where users really do need to enter other characters. What about hyphens in people's names? What if you're running a website like Stack Overflow and you need to allow users to enter actual code samples in their comments?

Surely there's a better way?!

Use Prepared Statements#

044. Use prepared statements.

We want to use something called "Prepared Statements" to solve this problem. This is one of the most surefire ways to protect against SQL injection.

Prepared Statements?#

045. Prepared statements? Reference

But what are they? Well, they're essentially an SQL template, where user values are substituted at runtime and can't modify the query. There's an added bonus here where this can also improve performance, as you can execute similar queries repeatedly with high efficiency.

Prepare#

046. Step 1: Prepare.

Let's look at how prepared statements work. The first step is to "prepare" the query. This is where you define how your SQL statement will look, but the value is left blank, and instead you define a "placeholder" (there are other names that can be used here too).

Optimize#

047. Step 2: Optimize.

The query is then sent to the database management system, where it’s compiled and optimized, but not executed (since it doesn't have any values yet).

Execute#

048. Step 3: Execute.

The final step is when we come to execute the query. We "bind" the placeholders to a real value (or variable), and then tell the DBMS to execute the query with those values. Since the DBMS has already compiled the query, the user values cannot be interpreted as SQL, they can only be used as a value. Which is exactly what we want to stop SQL injection.

Benefits#

049. Benefits.

Because the parameter values are transmitted later using a different protocol, they don't need to be correctly escaped. Assuming the original statement template is not derived from external input, SQL injection cannot occur.

We also get some optimization benefits from using this method as it’s compiled once on the DB and executed many times with different data.

Prepared Statement Example#

050. Full prepared statement example.

Here's a quick pseudocode example of how it would all look when tied together.

SQL Injection - Additional Reading#

051. Additional Reading.

SQL injection is one of the more common vulnerabilities that exist, and it can be devastating if exploited. There's lots of nuance involved and lots more I could have talked about. But since we don't have all day, here are some additional reading links if you're interested in understanding more about SQL injection and how to exploit it. The cheat sheets are particularly eye-opening.

• https://pentestmonkey.net/cheat-sheet/sql-injection/mysql-sql-injection-cheat-sheet
• https://www.owasp.org/index.php/SQL_Injection_Prevention_Cheat_Sheet
• https://www.sqlinjection.net/time-based/
• https://www.owasp.org/index.php/Blind_SQL_Injection
• https://ckarande.gitbooks.io/owasp-nodegoat-tutorial/content/tutorial/a1_-_sql_and_nosql_injection.html

Storing Passwords#

052. Storing passwords.

In the first SQL injection example, I showed passwords that were passed to an SQL query in plaintext. That's obviously a very bad way to store passwords. I also talked about how to crack passwords in the previous training. So what's the proper way to store passwords?

Hashing#

053. Hashing. Reference

First, let's quickly reHASH (get it?) what we covered in the last training.

A hash will take a value, and give you some seemingly random gibberish as an output. Given the same input, it will always give you the same output.

One-Way#

054. One-Way. Reference

They are essentially a one-way mathematical function. The idea is that it's very easy to take a value, and get a repeatable hash of it. But it's very difficult to have just the hash and reverse it back into the password.

Imagine trying to multiply two large prime numbers, multiplication is pretty easy and that wouldn't take too long. But what if you had the final number, and you had to discover which two prime numbers were multiplied to get it? That's going to be much harder to do. That's pretty much what's going on here.

So once you have a hashed value, you essentially can't get back to the password, unless you already have a mapping from hash back to password.

Rainbow Tables#

055. Rainbow tables. Reference

This is what a rainbow table is. A precomputed list of hashes back to their original value. It allows you to quickly reverse hashes. You can get rainbow tables for all of the common hashing algorithms in use. I used SHA-1 in all my examples from the last training.

Salting?#

056. Salting?

I said in the previous training that there's something called "salting" which can protect against this type of attack. So what is this "salting" stuff?

Salt Definition#

057. Definition of a salt. Reference

A salt is some random data that's appended to a password before it's hashed, that's different every time. That means a salt should never be re-used, and shouldn't be the same for every user on your site (I'll talk more about this in a moment).

It's very important that the salt is different every time you create or change a password. Why?

Salting#

058. Salting. Reference

Let's look at a typical example of how it works. I would take the user's password, append a random piece of data to it (that's the salt), and hash it the same way you would have hashed the password.

Salting (2)#

059. Same password, different hash. Reference

Where this really comes into play is when you have two users with the same password. Since the salt is going to be different every time, that means two users with the same password will now get a completely different hash result at the end of it all.

Database#

060. Database.

In our database, we would store the salt, along with the resulting hash. We store the salt so that we can still verify that a user has entered the correct password. So why bother with a salt at all? If an attacker has access to it, how does this help us?

Well, every one of the users in this database has exactly the same password. But because of salts, the resultant hash is different for everyone. This means you can no longer use a rainbow table to crack these passwords, since you'd have to make a custom rainbow table for each user (at which point you may as well just brute force things).

Rainbow Tables Infeasible#

061. Rainbow tables are infeasible thanks to salting.

Thanks to using a salt, we've now made rainbow table attacks completely useless. This is why the salt must be different for every single password, otherwise it defeats the point.

Salt is Public#

062. Salt is public.

You’ll notice that the salt was in plaintext in the database I showed, and that’s exactly as it should be. Salts are not secrets and should not be treated as such. They’re there to make rainbow table attacks infeasible, nothing more. Remember, that what we're doing here isn’t encryption, the salt is not a secret key or anything like that.

Pepper?#

063. Pepper? Reference

There's also another piece of data that can be used to make password storage stronger. This is called a "Pepper", or I've sometimes seen it referred to as a "Site-wide Salt". A pepper is a piece of random data that's appended to every password, similar to how the salt was, however the difference is that the pepper is the same for everyone and isn't stored in the database. The pepper is treated as a secret and is kept on disk, or as part of whatever secret storage solution you happen to be using.

Why is a pepper useful? Well, in order to be able to try and brute force any stolen hashes, you’d need not only the database (which has the salt), but also the pepper. If you’ve stolen the database via SQL injection, it’s unlikely (but not impossible) you can also steal the pepper which is stored on disk. It can make it more difficult for an attacker, but it can also make it more difficult for you too. For example, rotating a compromised pepper can be a difficult challenge, and generally you just have to keep track of every pepper you’ve ever had.

Pepper#

064. Pepper.

So if you wanted to use a pepper, the process would now look like this. You take the password, append the salt, append the pepper, and then hash that. That would then be the result you store in your database.

Whether you use a pepper or not though, there's still more to do.

Slow Down#

065. Slow down!

We still have a problem...if someone is able to steal the database and the pepper, they are still able to brute force those passwords just by trying every combination. They can't pre-compute them into a rainbow table, but they can do them in realtime.

Why is this possible? Because hashing is too fast! So far I've only shown you general purpose hashing functions, MD5, SHA-1, etc. The original purpose of these algorithms is to quickly compute a value so that you can use it to check and verify the integrity of a file. In those cases, you want it to be as fast as possible. But they weren't designed for secure password storage, where speed means that an attacker can try more combinations more quickly.

Slow it Down#

066. Slow it down. Reference

So there's another step we need to make here. We want to slow down how quickly our hashing works. We typically don't like to slow things down as engineers, but in this case it's warranted.

If it takes 1 second for a user to be able to login, the user isn't really going to notice. But an attacker who wants to try to guess passwords, only being able to try 1 every second is going to make it very tedious for them.

The way we slow it down is by just running the hashing algorithm multiple times (called rounds). Each iteration should rely on the output of the previous one so that it can't be parallelized. We're not just spinning CPU cycles here, we are actually calculating something. This process is called "Key Stretching".

In this example, we're going to run the hash 100,000 extra times.

Key Stretching#

067. Why do key stretching?

Why do we do this? Well, because we’ve just made it 100,000 times harder for an attacker to brute force those passwords offline. Before they could try 100,000 passwords a second. Now they can only try one. For a user, this isn't a big deal, but for an attacker, we've just make it much more painful for them.

Key Stretching (2)#

068. Things get flipped.

Unfortunately, we still have a problem. Hardware is always getting better. What's true today, might not be true in a year's time. An attacker might break out some GPUs and instead of mining Bitcoin, maybe they'll start hashing passwords instead. So how can we keep up?

Adaptive Hashing#

069. Adaptive hashing.

We need to use something called adaptive hashing instead.

Adaptive Hashing Definition#

070. Adaptive hashing definition.

Basically, adaptive hashing is the ability to change the number of rounds as time goes on. But 'rounds' are a strange concept, the speed is based on how fast your CPU is. So let's put this into other terms.

Cost#

071. Cost. Reference

Instead, what we really care about is the cost, or effort involved. There are algorithms where you can specify a "work factor", or "cost" value. Since what we really care about is how much time and money it's going to take an attacker to crack your passwords.

There's a great table here showing the different costs associated with cracking passwords using various different algorithms. PBKDF2, bcrypt, and scrypt are all purpose-built password hashing algorithms. They were designed specifically for passwords, so unlike MD5 or SHA-1, they are purposefully slower.

They also do pretty much everything I just talked about automatically. Salting and key stretching are built-in.

Use Bcrypt#

072. Use bcrypt.

So the key takeaway here is to use an adaptive hashing algorithm specifically designed for storing passwords. bcrypt, scrypt, or PBKDF2 are all good choices. We use bcrypt.

It's also important to reevaluate your "cost" value every so often. It's no good just setting a cost factor and leaving it for years, because eventually hardware will catch up.

Bcrypt Example#

073. Bcrypt example.

Here's an example of how to use bcrypt in Ruby. That $ delimeted string it gives you has all of the necessary information as part of it. The algorithm being used 2a, the cost/work factor 13, and a salt and resultant hash. So it takes care of all the details for you.

Storing Passwords - Additional Reading#

074. Additional reading.

If you want to know more about these algorithms, here are links to their wiki pages.

• https://en.wikipedia.org/wiki/Bcrypt
• https://en.wikipedia.org/wiki/Scrypt
• https://en.wikipedia.org/wiki/PBKDF2

Encryption#

075. Encryption.

OK, it's time for our next topic, encryption.

Encryption Definition#

076. Definition of encryption.

Hopefully you should all be familiar with what encryption is, but if not, encryption is about encoding information in such a way that only authorized parties can read it.

Encryption Comic#

077. XKCD on encryption. Reference

The biggest problem with encryption is that it’s very easy to convince yourself your encryption is great. It's easy to come up with an encryption scheme you yourself can't break. It doesn't mean that no one else can.

Don't Write Your Own#

078. Don't write your own.

For this reason, the best encryption advice is to never write your own encryption. Ever (Well.. unless it's actually your job). Leave it to the experts who do it for a living. Use a library, or pre-defined encryption algorithms. This is one area you want to very much leave to the professionals.

Encryption Types#

079. Types of encryption.

There are various different types of encryption, each suitable for different tasks.

Encryption Types Definitions#

080. Plain English.

Let’s try and translate these into plain English.

• Symmetric/Asymmetric refers to whether the key used to encrypt and decrypt the message is the same or different. So if the same key can both encrypt and decrypt the data, it's symmetric. If they're different, it's asymmetric.
• A block cipher is a type of encryption where the data is encrypted in chunks. Most of the encryption algorithms you've probably heard about are block ciphers; DES, AES, etc. They also have various different modes of operation, CBC, GCM, etc. I'm not going to go into all the different modes, since we'll be here all day, but you can read more about them on wikipedia.
• Public/Private key encryption is where you have a private key, and everyone else has a public key. This allows anyone to encrypt a message to you, that only you can decrypt. It also allows you to sign pieces of data, and everyone else can verify that you were the one who signed it.
• Stream ciphers are where the data is encrypted "on-the-fly" as a raw data stream, rather than in blocks like a block cipher.

There are various details and differences between types, and a lot more examples. This is a huge field of study, and as much as I would love to cover it all, we need to stop somewhere. So these are the main types you are likely to come across.

We need to use encryption for two important things...

Encryption in Transit#

081. Encryption in transit.

The first is called "encryption in transit"...

Encryption in Transit (2)#

082. Encryption in transit.

The idea is that we want to be sure that if any of our communications are intercepted, they cannot be read. Importantly, we also want to make sure they can't be read in the future even if our keys get leaked. So if someone intercepts our communications now, but doesn't yet have the key to decrypt it, they might store it on the off-chance they do get the key later, then go back and decrypt it when they do. You want to use a technique called "Perfect Forward Secrecy" to prevent that from happening.

Everything at PagerDuty is encrypted in transit automatically, we use HTTPS/TLS for all endpoints, and all other communication is encrypted by IPsec in transport mode by default. You need to try very hard to send something unencrypted around our infrastructure.

Encryption at Rest#

083. Encryption at rest.

The next main use of encryption is called "encryption at rest"...

Encryption at Rest (2)#

084. Encryption at rest.

We want to be sure that if someone gains access to our systems, that they cannot access any data while it’s sitting around in our own infrastructure.

So we want to be sure that all information is encrypted when it's stored on disk or in S3, etc. You may have heard that your laptops all have "Full Disk Encryption" enabled, this means that your laptops are encrypted at rest. If someone were to yank out the hard disk from your laptop in order to steal information, they wouldn't be able to, since the disks are encrypted.

Should I Encrypt That?#

085. Should I encrypt that?

So what information should you be encrypting at rest and in transit? What’s the most sensitive data we have? (Spoiler alert: it's customer data).

Base64 a Password#

086. You wouldn't base64 a password.

You wouldn’t base64 a password, so don’t store customer data in the clear either.

Data Classification#

087. Data classification.

A quick reminder of our data classification policy. We have three distinct types of data at PagerDuty.

• General Data is anything that is intentionally available to the public. The key being intentionally. If something gets leaked, it doesn't automatically become general data.
• Business data is anything used to operate our business. Examples would be our employee list, payroll information, etc.
• Customer data is any data provided to us by the customer.

Customer data is by far the most valuable and protected data we have at PagerDuty, and is subject to our most stringent security rules.

Data Handling#

088. Data handling.

How we handle data depends on what type of data it is. For example, general data does not need to be encrypted, as it's considered public. All other types of data should always be encrypted in transit and at rest however.

You can view a more detailed description of our data handling policies on the wiki.

Customer Data#

089. Customer data.

So, customer data should always be encrypted in both transit, and at rest. Always.

AWS Encryption#

090. AWS encryption. Reference

Our automated tooling takes care of most of this for you, but there are some edge cases where you may need to select a specific option to enable it yourself. Especially if you're using new services or tools in AWS, please make sure they're encrypted. It’s usually just a single click, and is super easy to enable.

We typically get paged if you create a new unencrypted resource, and your newly created object will likely get automatically terminated and you’ll have to create it again. So save yourself the hassle and always enable encryption.

Third-Party Systems#

091. Third-party systems.

What about third-parties that we need to give our data to? Do they need to have encryption too? Yes, absolutely! Third-parties need to follow the same rules as us. If we encrypt our customer data but a third-part doesn't, that's just as bad as us not doing it. This is why we perform vendor risk assessments, to make sure that our vendors have the necessary security requirements in place to be able to handle our data. If they don't, then we can't use them.

So please make sure to get a vendor risk assessment completed before you try and start using a third-party tool.

Encryption - Additional Reading#

092. Additional reading.

Like I said before, encryption is a huge area, and I only really touched the surface here. So here's some additional reading links if you want to learn more.

• http://www.networksorcery.com/enp/data/encryption.htm
• https://www.owasp.org/index.php/Guide_to_Cryptography
• https://gist.github.com/tqbf/be58d2d39690c3b366ad

Secret Management#

093. Caption.

Alright, next topic is Secret Management.

Secret Management Definition#

094. Definition of secret management.

What does that mean? Well, secret management is all about protecting and auditing access to secrets. We want to make sure that no one evil can look at our secrets, but that if they do we have a record of it. We also want to know who's looking at our secrets, even if they're allowed to.

Secrets?#

095. Secrets?

But what do we mean by "secrets"?

The best way to manage a secret is to not have one in the first place. But that isn't always possible. A decent rule of thumb is that anything with the words "token", "key", "password", or "secret" in the name should be considered a secret. That doesn't cover all cases, but most things fall under those categories.

ಠ_ಠ#

096. ಠ_ಠ

What we absolutely don't want is secrets hardcoded into our source code. It can be tempting to hardcode things like this, but it's never a good idea. The reason this is bad is that anyone with access to the source code gets access to the secret. Generally, the developers building the software don't need to know the secrets that are used to operate that software. Ideally, you want all that to be automated with humans never even seeing the secrets at all. But at the very least, you want the secrets decoupled from the source code.

Vault#

097. Vault.

So how should secrets be stored? You need a dedicated system for managing them. Just like how you use password managers to manage passwords, you should use a secret manager to manage secrets. We use Vault for secret management at PagerDuty, which allows us to securely store secrets, easily rotate them when necessary, and gives us a full audit log of when they were accessed, and who accessed them.

Vault UI#

098. Vault UI.

You don't need to interact with it on the command-line in a terminal, we have a UI for it installed in all our environments so that you can manage it visually instead. We have plenty of documentation about this on our wiki, so check that out if you're interested in the details.

Use Vault#

099. Use Vault.

The main takeaway here is that we should always be using Vault to store secrets for our applications or systems. Don't hardcode things or come up with any other strange methods of getting secrets into your applications.

Need The Password#

100. "I need the password for..."

On the subject of secrets, I occasionally see messages like this in Slack: "I need the password for..."

We should try and get out of this habit, as it not only assumes that we have a shared password for everything (which obviously isn't the case), but it's subconsciously encouraging passwords to be sent over Slack.

Very rarely is there a "password" for something that needs to be shared. We should all have individual accounts to things so that we have accountability. So the question should really be "I need access to..." instead. Even then, if a secret needs to be shared, it needs to be done the correct way.

Never Share Secrets Insecurely#

101. Never share secrets over insecure channels.

Remember that secrets should never be shared over insecure communication channels. Slack is an insecure communication channel, so is email. If there's a secret that needs to be shared, it should be in Vault, or in 1Password, or some other dedicated secure secret delivery tool.

Don't Share Secrets#

102. I'll say it again.

I'll say it again, because this really is important. Do not share secrets on Slack.

Slack Secrets#

103. Sharing secrets on Slack.

But what happens if you do it accidentally? As shown in this completely made up and totally not real scenario.

It can happen, you had something in your clipboard and paste without realizing. By the time you hit enter and spot it, it's too late.

Well, don't panic. We all make mistakes. You're not going to get into trouble. I've accidentally posted secrets in chat tools before, I'm sure most people have. What matters is what you do next...

Notify Security#

104. Notify the security team.

...you need to let the security team know immediately. We'll walk you through how to rotate the credential and get things back to normal. Now, we're not going to do the rotation for you, we're going to show you how to do it yourself. So don't just think you can pass off the work to us :)

Importantly, we will never name and shame you if you do something silly like paste a password into Slack. Never be afraid to come to us if you've accidentally done something that's not secure. As anyone who's already done this knows, we keep it to ourselves. We will usually do a post-mortem, but you will never be mentioned by name or any other identifier. We all make mistakes, and our job isn't to make you feel bad if you happen to make one too.

Secrets in Logs#

105. Secrets in logs.

Another part of managing secrets is to make sure they never appear in places they shouldn't. Logs are typically where this happens by accident. You enable some sort of debug-level logging, and all of a sudden tokens, API keys, and passwords start appearing in log files. Even worse, they're not passwords we control, so we can't just rotate them. They're likely to be customer passwords. Which means we'd have to notify those customers to change their credentials.

We have automation in place to redact anything that even remotely looks like a secret, but it's not perfect and you shouldn't rely on it.

Be Mindful of What You Log#

106. Be mindful of what you log.

Be mindful of what you're logging, and if it's a secret, don't log it. If you need to do any sanitizing, make sure it happens before the log is written to disk or uploaded to our centralized logging tools.

Secret Management - Additional Reading#

107. Additional reading.

If you're interested in learning more about secret management, our very own Max Timchenko wrote up a big overview of various secret managements tools. It's well worth a look.

• https://gist.github.com/maxvt/bb49a6c7243163b8120625fc8ae3f3cd

Cross-Site Scripting#

108. Cross-site scripting.

Our next topic is "Cross-Site Scripting", which you'll usually see abbreviated as XSS.

XSS Definition#

109. XSS definition.

So what is it? Cross-site scripting is all about injecting little bits of code onto web pages that will be viewed by other people. The injected code then gets executed by them without them realizing. It allows an attacker to steal pretty much anything if executed properly.

XSS Example (1)#

110. Awesome website.

Let's look at a quick example. Say I've got a website, where I'm storing some user-defined data. In this case, just a comment.

XSS Example (2)#

111. Enter a malicious comment.

A malicious user enters a comment that contains some JavaScript, which will popup a message saying "hello", and submits that comment to the site.

XSS Example (3)#

112. Uh oh!

If the site is vulnerable to XSS, whenever anyone visits that page, their browser will now show them a popup message saying "hello".

So OK, that's kind of annoying, but it's not really that dangerous is it?

Dangerous#

113. It's kind of dangerous.

Actually, yeah, it's pretty dangerous.

Stealing Cookies#

114. Stealing cookies.

Because an attacker could use this to steal all sorts of information, maybe even your cookies (We'll be talking more about what those are later), which means the attacker could login to the website as if they are you, and the website wouldn't know the difference.

If they can't steal cookies, they could surreptitiously steal information about your account instead, post comments as you, etc. Whatever JavaScript they can think of, they could execute as you.

Sanitizing Inputs#

115. Sanitizing inputs.

Some common advice for dealing with XSS is to make sure you're sanitizing all your inputs. That's generally good advice anyway, but it won't cover all cases for XSS and can still leave you vulnerable.

While validating you have an integer when you expect an integer is good, (or alphanumeric when you expect it, etc). It doesn't always work for other fields, like descriptions or comments. You'll end up stripping things that shouldn't be stripped. For example, what if someone wants to post code as a comment? Sometimes you really do need to present what the user entered verbatim.

Encode on Output#

116. Encode on output.

Instead, the only surefire way to protect against XSS is to encode all user information when it is rendered back to the user, such that it cannot be executed by the browser. All characters would be encoded so the browser displays them as is, and doesn't interpret them as raw code to be executed.

HTML#

117. HTML.

If our previous example was encoding on output, it would have looked like this in the HTML, and would never have shown the popup message to users. That's what we want.

Ember#

118. Ember. Reference

So how do you do this encoding? It depends on the language you're using. We use a lot of Ember here at PagerDuty, and in Ember you just need to wrap variables with {{ }}. But, add one more { } to the mix, and it will no longer encode your output. So you need to be very careful that you use the appropriate method.

HAML#

119. HAML. Reference

We also have lots of HAML, which similarly can encode output for us. In this case, just make sure to use = and not != when outputting a variable. The ! would disable the encoding and make it susceptible to XSS.

Encode User Data#

120. User supplied data should always be encoded on output.

The key takeaway here is that user supplied data should always be encoded when it's output back to a browser. Never let a user control what can be executed on your site.

Not Just HTML#

121. Not just HTML.

Even though it's what I focussed on here, it's not just HTML output that needs to be encoded. You can't just HTML encode output and put it anywhere, the way you encode needs to change based on where it is. If the user value is getting placed between some <script> tags, or inside CSS, then HTML entity encoding won't be enough to protect you. They all need to be encoded too, and each in their own special ways.

Use a Library#

122. Use a library.

So you'll want to use a library to do this for you. I have some warnings about using libraries and frameworks earlier, but that doesn't mean you shouldn't use them, just don't blindly rely on them.

XSS - Additional Reading#

123. Additional reading.

If you want to read more about cross-site scripting, here are some good links with more information. CSP or Content Security Policy is something I didn't get time to cover, but is another way to prevent XSS if implemented correctly. So I encourage you to read up on that.

• https://www.owasp.org/index.php/Cross-site_Scripting_(XSS)
• https://developers.google.com/web/fundamentals/security/csp/
• https://en.wikipedia.org/wiki/Content_Security_Policy

Cross-Site Request Forgery#

124. Cross-Site Request Forgery.

OK, next topic. Cross-Site Request Forgery, usually abbreviated to CSRF.

CSRF Definition#

125. Definition.

CSRF is all about tricking a user into performing an action they didn't want to perform (or didn't realize they were performing). It can be a little tricky to wrap your head around, so let's start with a quick example.

Pretend To Be An Attacker#

126. Let's pretend to be an attacker.

Let's start off by pretending to be an attacker.

Forum#

127. Forum.

We have a forum where people can post comments. Here you can see a user called Rich has posted some comments, and now the attacker (that's us) is going to add one too.

Posting an Image#

128. Posting an image.

We're going to post an image, but rather than link to an actual image file, we're going to set the link as the URL to logout instead.

Switch To User#

129. Switching to the user.

Let's change perspectives now and view things from the user's point of view.

Rich#

130. Rich.

When Rich visits the page, they're going to see a broken link. But in the background, their browser made a request to get the image. Let's see what's happening behind the scenes...

Behind The Scenes#

131. Behind the scenes.

Behind the scenes, Rich's browser has seen that the user wants to load an image. It knows the website it's requesting an image for, and has some cookies for that site (we'll talk more about these in a moment), so it makes the request and sends along the cookies.

From the website's point of view, the user, from their own browser, with all their authentication information, just asked them to logout. So what does it do? It logs them out!

Logged Out#

132. Logged out.

So the outcome is that Rich has been logged out from the site, just by visiting an unrelated web page. Now logout isn't necessarily a nefarious action, but this could certainly become annoying for users.

Do it as a POST?#

133. Do it as a POST request?

When demonstrating CSRF using GET requests, one of the frequent comments is "Couldn't we do this as a POST request to stop it?", since the user will then need to actually click a button instead of just visiting the page. In the previous example that may have helped, but only because of the example I was using. So let's look at a different example of CSRF against POST requests.

Delete Account#

134. Delete account.

In this example, we've crafted a form which will delete an account on example.com. We've put the text of the button to say "Click here to win a prize!". All we have to do is trick a user into clicking the button, which can be done a variety of different ways using social engineering. We can present the button anywhere, it doesn't have to be on the same website. We could host it on an internal wiki if we wanted. Once they click it though, if the example.com site is susceptible to CSRF, then their account would get deleted.

Panic#

135. Panic.

That's pretty scary. You click a random button on a website, and an account you have on a completely unrelated website gets deleted. It's an extreme example, but it gives you an idea of how scary these types of vulnerabilities can be.

Synchronizer Token#

136. Synchronizer token.

So how do we protect against CSRF? We use something called a "synchronizer token", or "CSRF token". This is a hidden field you include on every single form for a user, which contains a value that's private to a particular user's session.

Token Should Be#

137. The token should be...

We want these tokens to be unique per user and session, be a large random value, and generated by a cryptographically secure random number generator. Basically, we don't want these tokens to be at all guessable, or shared with anyone else.

Server-Side#

138. On the server-side.

So how do they work? Well, once you've assigned one of these tokens to a user's session, you make sure to include it on every form that's presented to a user, where it will contain the value for that specific user.

Then on the server-side, whenever you receive a request from that user, you verify that a token is present, that it's the user's token, and that it hasn't expired. (You can also make the tokens single-use, and validate it's not already used. That's a bit more complicated though and not always necessary).

If any of these checks fail, then the request should be aborted, since it could be a potential CSRF attack.

Rails#

139. Rails. Reference

Thankfully, all of this logic is usually built into most frameworks. In Rails all you need to do is add protect_from_forgery to your controllers. But at least now you know what it's doing behind the scenes!

Revisit The Example#

140. Revisiting the earlier example.

Now if we revisit our earlier example, you can see how the attacker would no longer be able to craft a form that would work, since they would have no way of knowing the correct token without already having access to the user's session (which if they did, it would be game over anyway). So we've made it impossible for them to perform a CSRF attack against those requests.

Always Use CSRF Tokens#

141. Always use CSRF tokens.

So the key takeaway here is to always use CSRF tokens whenever you're performing state changing operations. That is essentially anything that's not a "READ".

GET#

142. Don't use GET for state changes.

Actually, quickly on that subject, also make sure not to use GET requests for state changing operations. I remember a fun story that made the rounds a few years ago where there was an administrative panel which allowed you to delete rows from a database. The "Delete" links were all GET requests. A user visited the admin overview page, and their browser went a pre-fetched all of the links on the page in order to speed things up. But of course, since it followed the links, it went ahead and deleted the entire database. Fun times.

Clickjacking#

143. Clickjacking.

One final type of attack I wanted to cover, that's similar to XSS and CSRF is something called "Clickjacking".

Clickjacking Example#

144. Clickjacking example. Reference

This is where you render a website in an iframe, put your own stuff on top of it, but pass through any clicks the user makes. So for example, you render eBay in the background on a product you're trying to sell, and line up the "Buy It Now" button with a fake button you've added that says something else. If you trick the user into clicking the page, and they're logged into eBay, it would purchase your item and send you some money.

Another method is moving a window or showing a popup just as you go to click, etc. So you end up clicking something else.

X-Frame-Options#

145. X-Frame-Options.

Thankfully this is pretty easy to defend against. There's a HTTP header we can include in our responses which tell a browser whether it's allowed to render our website in a frame or not. The two main options which prevent clickjacking are SAMEORIGIN, which means it can only be loaded into a frame if we're the ones doing it. Or DENY which entirely prevents the site from loading in a frame.

Every Logged In Page#

146. Every logged in page.

So the key here is to make sure this header is set to SAMEORIGIN or DENY on every logged in page.

CSRF - Additional Reading#

147. Additional reading.

If you're interested in learning more about cross-site request forgery, here are some links you can check out.

• https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)
• https://en.wikipedia.org/wiki/Cross-site_request_forgery

Account Enumeration#

148. Account Enumeration.

Next topic is Account Enumeration, and this is a nice short one.

Account Enumeration Definition#

149. Definition.

So what's this all about? Account enumeration is where you can extract a list of users or customers from a website. It's very useful for social engineering, but can also lead to security issues too if you're able to extract email addresses.

Let's take a look at some quick examples.

Email Login Form (1)#

150. Login with bad password.

Let's say we have a login form, where I try and login as myself with a bad password. It takes 3 seconds before it comes back with an error message.

Email Login Form (2)#

151. With a different email.

I try again with a different email address, but this time it comes back almost immediately with an error.

What can we infer from this?

Confirming Emails#

152. Confirming emails.

We can infer that the first is a user with an account on this site, and the second isn't. What's happening in the background is that in the first instance, the user was found, and the website attempted to hash the password to validate it. That took 3 seconds before it knew the password was wrong and showed an error.

In the second case, the website couldn't even find the user, so never attempted to hash the password, it just failed immediately.

Little things like this can give away lots of information.

PagerDuty Account#

153. PagerDuty account.

Let's look at another example. Let's say I visit the subdomain for a PagerDuty account I suspect exists. It loads the login page. OK great.

Not a PagerDuty Account#

154. Not a PagerDuty account.

But now if I visit one that doesn't exist, I get a different experience.

Confirming Accounts#

155. Confirming accounts.

From this I can infer that the first company has a PagerDuty account, but the latter doesn't. Now this obviously isn't 100% foolproof, nothing says someone has to chose a subdomain that matches their company, in fact a lot don't. But we've actually had competitors exploit this particular behavior in the past in order to identify companies that use us, and then contact those existing customers asking them to switch to their own product! (I know, cheeky right?!)

So why is it still this way? Surely the fix is easy, just show a login page every time. That would indeed solve this issue, but it also presents other problems, as we discovered first-hand. If a customer mistypes their subdomain, then attempts to login, they get login failures. The first assumption is that their account is disabled or PagerDuty isn't working, and they reach out to our support team. Showing a clear error that the subdomain is wrong stops us getting these types of requests, and lets users know that they got the wrong subdomain.

The point I'm trying to make here is that there's sometimes a trade off between security and having a better user experience, and sometimes the better user experience makes more sense.

Preventing Enumeration#

156. Preventing enumeration.

We can infer lots of little bits of information based on how we display things back to users. It's very difficult to fully prevent these types of problems. There's usually always at least some way to figure out if an account is real. For example, another common trick is to try a password reset, and the website will tell you if they sent the email or not, etc.

But in general, you want to make sure failure paths still follow a similar flow. In the user login case, hash the password even if you already know the user is wrong, so it takes roughly the same amount of time.

If you can, you also want to try and avoid any boolean tests for if an account or customer exists. As mentioned before though, these usually exist to make the user experience better, so the trade off can be worth it. For example, you can test if a Google account is genuine by trying to login. If you get to the password screen then an account exists, if it doesn't you'll get an error saying the account doesn't exist.

Be Mindful#

157. Be mindful.

But as a general rule, just be mindful of the type of data you might accidentally give an attacker insight into.

Account Enumeration - Additional Reading#

158. Additional reading.

If you want to know more about this and why it's an issue, here are some pages you can visit.

• https://owasp.org/www-project-web-security-testing-guide/latest/4-Web_Application_Security_Testing/03-Identity_Management_Testing/04-Testing_for_Account_Enumeration_and_Guessable_User_Account
• https://blog.rapid7.com/2017/06/15/about-user-enumeration/

Session Management#

159. Session Management.

OK, you've heard me talk about cookies a few times. So now we're going to talk about Session Management, where I'll go over just exactly what these "cookie" things are.

Session Management Definition#

160. Definition.

First, what do we mean when we talk about "Session Management". Well, we're talking about being able to identify a user over multiple requests. We want to know if the person we're talking to now, is the same person we were talking to a moment ago.

HTTP is Stateless#

161. HTTP is stateless.

This is an issue because HTTP is a stateless protocol. What does that mean exactly?

Stateless#

162. Stateless.

In a world without session management, each HTTP request happens in complete isolation. So let's say a user goes to login to a website. They say "Hi, I'm Bob, here's my password". The website sees their password, checks that it's them, and replies in the affirmative.

The user might then want to take a look at their profile. But without any session management, this new request has no previous context whatsoever, so the website would have no idea who they're talking to.

Cookies#

163. Cookies.

To solve this problem, we use something called cookies. No, not this type of cookie.

What are cookies?#

164. What are cookies?

Cookies are essentially just a piece of data that a server asks the client to remember. The client is then supposed to send that piece of data back to the server if it ever makes a request to it again. There are a few caveats involved, where there are some additional checks that take place, but that's essentially the gist of it.

Session Example#

165. Session example.

So let's look at our original example, but in a world where cookies exist.

The initial request from the user is the same, they tell the website who they are and that they want to login. The server checks that their password is correct, but before they respond to the user, it will create a "session". It will create a record of who the user is and that they're logged in, and will assign a unique identifier to this "session". The server will now reply to the user as before, but will include the cookie in its response, telling the user's browser to remember that session identifier and to use it on all subsequent requests.

The client will then remember that information and store it in the browser.

Session Example (2)#

166. Showing the profile.

Now when it comes to the second request, where the user wants to see their profile, the browser will now contact the server, but this time it will include that unique identifier it was told to include. When the server receives the request, it will see this ID, and look it up in its little database of sessions. If it finds a match, it can now see all the state about that user it needs to. It knows the user logged in previously, and then knows whose profile it should return. It can now send the response back to the user with the data they asked for.

The user is happy, since now they can finally see their profile.

User Identification#

167. User identification.

Some things to note about the user identification. We want to be sure to only store the unique session identifier in the cookie. The session data itself should be kept on the server side. Client-side session cookies (this is cookies where you store the entire session state in them) should be avoided, since it can be very tricky to remotely revoke them. The control is entirely on the user's side. You don't want your users to be able to manipulate their session data, since they could start telling you they're someone else for example. Usually you would add some sort of integrity check to the client-side session cookie so you know it's not been tampered with, but that still doesn't get around the problem of revoking it.

Store Session on Server#

168. Store all session data on the server side..

The main takeaway here is to never give the user the option to manipulate sessions, always store session data on the server-side.

Session Hijacking#

169. Session hijacking.

So what are some of the problems you can have with session management? The main problem is something called "Session Hijacking".

Session Hijacking Info#

170. Session hijacking.

This is where an attacker can take over the session of another user. This can happen in several different ways. Either they were able to steal your session identifier, so they can tell the server that they're you. Another way is by just guessing session identifiers (this is why they should be random!), another way is by manipulating cookies that weren't stored properly. If a cookie was made accessible more widely than it should have been, an attacker might be able to manipulate it to steal your session, or have you change from your own session to their session...

Session Fixation#

171. Session fixation.

Which brings me to another attack, session fixation. This is where an attacker is able to start a session themselves, then have you use that session without you realizing. Then once you login, they get logged in as you too. This used to be really easy when session IDs where passed as GET parameters, which is why you no longer see that (at least you shouldn't!)

Cookies Are User Data#

172. Cookies are user-supplied data.

One of the key things here is that cookies are user-supplied data. You should never blindly trust them, treat them like you would any other user input. You should always verify the authenticity and integrity of a cookie before you start using it, especially when using for session management. So how do we secure sessions?

Verifying Session#

173. Verifying session.

The first step is to verify the session data. This is as simple as adding a few extra pieces of information to the session data you store on your side, and then verifying things against that when a user says it's their session. You want to add an expiration date, so you can expire the session if it's too old. You also want to confirm that you actually created the session, and it's not just a random ID an attacker is trying.

You can also do something called a loose IP check, where you verify the first few octets of their IP address match what the user was using (this would spot things where a user suddenly connects from a different country, etc). There are some problems with that though, where it can give you some false positives, so use with caution.

All of these checks allow you to kill potentially suspicious sessions ahead of time. It catches the low hanging fruit, but isn't a 100% way of stopping attacks.

Protecting Session IDs#

174. Protecting session IDs. Reference

We also want to protect our session IDs so that we limit the potential for them to be stolen by an attacker. As already mentioned, we want to be sure they're both unique and random. If an attacker can guess or reuse a session identifier, that can lead to the session being taken over.

When passing a cookie to the client, we want to be sure we set the secure and httpOnly flags on those cookies. This ensures that the session ID is only ever sent over a secure connection, and cannot be accessed via a client-side script (reducing the risk of it being stolen if someone exploits an XSS vulnerability for example). We also want to be sure we specify a domain, so that the cookie won't be shared with anyone else but us.

The final thing is to always make sure you generate a completely new session ID and send that to the user if their privileges ever get elevated (i.e they login). This helps to prevent session fixation, since the user would be in a new session once they've logged in.

Protecting Session Data#

175. Protecting session data.

We also need to protect the actual session data we're storing. As I mentioned earlier, we want to be sure we're storing it all server-side. We want to make sure that we have an expiration date in our session data itself, we shouldn't be relying on a user's cookie to expire to end the session, we need to make sure we proactively expire it on our side. Remember that a cookie is user supplied data, they can change the expiration date if they want.

Finally, when a user logs out, we want to completely destroy their session on the server side, so there's no chance of it accidentally keeping a user (or attacker) logged in.

Never Trust User Input#

176. Never trust user input.

As with most of the vulnerabilities today, the common theme is to never trust user input. Cookies, input fields, anything the user controls should never be trusted. Don't forget that this can include data a user has already stored in our database!

There are lots of examples where users can do things you don't expect.

Example (1)#

177. "Using JavaScript as your only form of input validation".

This example is titled "Using JavaScript as your only form of input validation".

Example (2)#

178. "A secure bike storage solution".

Here's a "secure" bike storage solution. Users will always find ways to bypass checks you have in place.

Example (3)#

179. SQL license plate.

You just can't trust users. They'll try anything.

Session Management - Additional Reading#

180. Additional reading.

As is has become tradition in each section, here are some links you can read if you want to know more about session management.

• https://www.owasp.org/index.php/Session_Management_Cheat_Sheet
• http://www.browserauth.net/channel-bound-cookies
• https://tools.ietf.org/html/draft-west-origin-cookies-01
• https://tools.ietf.org/html/rfc5929

Permissions#

181. Permissions.

OK, I know we've been here a while, I only have two more sections left though. This is one is a quick refresher on permissions.

Piping to Sudo#

182. Totally legit.

What’s wrong with this picture? You're sending some random script over an insecure channel and executing it with root permissions on your shell. Hopefully you can all see why this isn't an ideal situation.

With Great Power#

183. With great power...

Remember, with great power comes great responsibility. Don't use sudo for things where you don't have to. Better yet, don't have the power at all if you don't need it. Use it wisely if you do.

Be Mindful of Your Power#

184. "This is too much power for one person".

You want to always be mindful of how much power you have, and the environment you're wielding it in. If you just need to read some data, try using read-only permissions instead of something that gives you complete administrative access.

Revoke Privileges#

185. Revoke privileges you don't need.

And revoke privileges if you don't need them. Don't just acquire an endless list of permissions you don't use. If you don't need sudo on a system, revoke it. If you don't need access to AWS Production, revoke it. The most secure accounts are the ones which don't exist in the first place.

Be Mindful#

186. Be mindful.

If you're running some reports on a DB, try a read-only user instead. If you're deleting things from S3, don't use a role that has permissions to delete an EC2 instance, use a role that is limited just to the bucket you need, etc.

Least Permissive Access#

187. Least permissive access.

The key takeaway here is to always use the least permissive access you can to accomplish what you need. This reduces the changes of accidentally doing something you didn't want, and also means if an attacker gets the credentials that they are also limited in what they can do.

Buffer Overflows and Other Classics#

188. Buffer Overflows and Other Classics.

OK, so I know I've been speaking to you all for a while now, but I wanted to end with something that might not be super relevant in your day to day roles, but which I thought was interesting nonetheless. I wanted to cover some classic vulnerabilities that aren't necessarily web app related, just a bit of fun and history.

Buffer Overflow (1)#

189. Contrived buffer overflow example.

A buffer overflow is one of those vulnerabilities you hear about a lot. But what actually is it? Essentially it's when a piece of data is allowed to be stored somewhere it shouldn't be able to access.

Let's look at a super contrived and completely silly example. Suppose I have an application where you can enter your name, and you get $5,000. It stores this information in a ledger. So in this case, it might be stored as a line that contains your name, and the amount I'm going to pay you.

Buffer Overflow (2)#

190. Contrived buffer overflow example.

Suppose I now enter my name, but then a load of null values, followed by a much higher amount of money. If the ledger is susceptible to a buffer overflow, this would get stored and would look like I'm now owed that amount of money instead. We've effectively overwritten some data we shouldn't have been able to.

Example Buffer#

191. Example buffer. Reference

So why does that matter? Well, let's switch to a much more realistic example, where instead of some text ledger, we're doing it with memory. Here we have an example buffer.

Peter#

192. "Peter". Reference

Under normal operation, you'd type in some text and it would be stored in memory. In this case, the string "Peter" (null-terminated).

Overflow#

193. There's an overflow. Reference

But if there's a buffer overflow vulnerability, we can overwrite the things after the buffer. Things like the return address, which tells the processor which bit of memory to start executing next.

Return Address#

194. Return address. Reference

This allows us to write some processor instructions into memory, then point the return address to where we just wrote those things to. The instructions we just stored in memory will then get executed! This means we can pretty much do whatever we want on a system (get a root shell for example). But how do you know where to point the return address to? If you don't get it exactly right, it won't execute your code.

NOP Sled#

195. NOP sled.

For that, we can use something called a NOP sled.

NOPs#

196. Buffer of NOPs. Reference

A NOP sled is where you fill the buffer with NOPs (0x90 in Intel x86) before the code you want to run. You then just need to get a return address that goes to anywhere in that sequence of NOPs. You don't need to find an exact position, just anywhere in your sea of NOPs will work, since then the processor will just execute nothing all the way until it hits the code you wanted to execute. Very cool.

Shellcode#

197. Shellcode. Reference

If there's a buffer overflow vulnerability, you can essentially use this technique to get a root shell quite easily. The post linked here has lots of examples of this technique, called "Smashing the Stack", and I highly recommend giving it a read. The code example here is taken directly from there. Remember, don't just run random code from the internet, take a look at the article first and understand what's going on.

Path Traversal#

198. Path traversal. Reference

Another classic vulnerability is called "path traversal". This is where you can escape out of the directory a web server is running from in order to access files from elsewhere on the filesystem. For example, you might be able to dump the list of users and their password hashes from a host.

One of our dependencies had a path traversal vulnerability a while back. It's important to remember that our dependencies can have vulnerabilities too, which is why we do vulnerability scanning on all the dependencies we use.

Side-Channel Attacks#

199. Side-channel attacks. Reference

Side-channel attacks are where you can use some information that's not normally part of an encryption process to infer some data and potentially compromise that process. There are a few different types, with the main ones being timing attacks, power analysis, acoustic cryptanalysis, and data remanence. Let's try and explain these in plain English.

Side-Channel Attacks Explanations#

200. Side-channel attacks in plain English. Reference

• A timing attack is where you can use the timing of certain operations to infer data about what's happening. For example, if I know a multiplication takes twice as long to execute as an addition. I can perhaps infer the mathematical operations a processor is performing when it decrypts some data, and gain some insight into how the decryption works.

• Power analysis is really cool, it's where you can analyze the power consumption of a processor (or anything really), and figure out what it's doing behind the scenes. There's a technique called "Differential Power Analysis", which is where you can statistically figure out what is going on by taking lots of power measurements.

• Acoustic analysis is also really cool, and that's what the YouTube link at the bottom of this slide is an example of. This is where you can analyze sounds to figure out what is going on. For example, maybe I could place a microphone near you and figure out your password from the different sounds the keys on your keyboard make.

• Finally data remanence, where you can recover sensitive data after it has supposedly been deleted. This is why you can find lots of "Secure Erase" tools which will overwrite your hard disk multiple times with random data, to prevent anything being able to be recovered later.

Additional Reading#

201. Additional reading.

If you're interested in some security history, or other classic vulnerabilities, there's some great material at these links.

The story of the NSA's involvement in the development of DES is particularly interesting. Back in the 70's they basically stepped in and slightly modified DES before it was released and wouldn't tell anyone why. Naturally people were a little suspicious. 30 years later a new type of attack called differential cryptanalysis came to light, but it was found that DES was surprisingly resilient to it, it turns out due to the changes the NSA had made. They knew about the attack and wanted to make DES resilient to it, but couldn't tell anyone much more without revealing they knew about differential cryptanalysis.

There's another fun story about how DirecTV managed to counteract some hacked satellite TV smart cards in what was called the "Black Sunday Hack". It's really interesting how they did it, by basically loading some code piece by piece over 60+ updates, which individually seemed innocuous, but when activated would make any hacked card completely unusable. They chose to activate the hack a week before the Super Bowl.

Anyway, lots of interesting reading!

• https://en.wikipedia.org/wiki/Data_Encryption_Standard#NSA's_involvement_in_the_design
• https://simson.net/ref/1994/coppersmith94.pdf
• https://en.wikipedia.org/wiki/Differential_cryptanalysis
• https://en.wikipedia.org/wiki/Power_analysis
• https://www.nsa.gov/news-features/declassified-documents/cryptologic-histories/assets/files/cold_war_iii.pdf
• https://www.theregister.co.uk/2001/01/25/directv_attacks_hacked_smart_cards/

Would you like to know more?#

202. Would you like to know more?

And that's it! I know that was a huge amount of information to cover in such a short amount of time. But I hope that you all learned something useful, or at the very least had a good laugh at some of the jokes.

Recommended Reading#

203. Recommended reading.

Security is a huge topic, and there are lots of specialty areas. Here are a few books I thought would be most relevant to engineers, and ones I can wholeheartedly recommend. It's well worth using some of your training budget on them if you're interested in learning more.

Contact Security Team#

204-208. Caption.

Questions?#

209. Questions?

So with that, I'll open it up to questions. Thanks very much!

Image credits#

210. Image credits.