Profiling Code; The Microsoft Way

Posted by jasper22 at 17:24 | Thursday, January 24, 2013

A colleague of mine asked me how he can find out why his laptop does consume 30% CPU in the SYSTEM process after it did wake up from hibernate. That is easy I replied: Open Process Explorer select the SYSTEM process, double click it, select the thread tab, sort by CPU and double click on the first thread which does consume so much CPU to get the call stack.

A few days later I got a mail back that it did not work. OK, could be some elevation problem. I did try it on my machine and sure enough access denied also here. Ups it is not so easy after all. A little Googling around did lead to a very good article from Mark Russinovich dealing with the exact same issue. Mark did use Kernrate which seems to work still on Windows 7 but it does look rather unsupported. A few other articles indicated that xperf should also be able to do it. So I did look where to get the latest edition of XPerf. It turns out that it is part of the Windows 8 SDK and the Windows Assessment and Deployment Kit (ADK) für Windows® 8. You do not need to download GBs of stuff. Only select the Windows Performance Toolkit and uncheck all other stuff. That gives you a nice ~50MB download which will install it to

%PROGRAMFILES(X86)%\Windows Kits\8.0\Windows Performance Toolkit

Besides the well known Xperf tool I have written several years ago some new stuff was added. Recording performance with the latest version of the Windows Performance Toolkit has never been so easy. There is a tool called WPRUI.exe which allows you to record a trace session with a nice GUI. An interesting side note is that this tool is missing in the Windows 8 SDK delivered with Visual Studio 2012. If you download the latest version again you get the GUI tool as well. It does seem to have full support from MS and they did even localize the UI which is rather unusual for such low level tools.

Nokia 'hijacks' mobile browser traffic, decrypts HTTPS data

Posted by jasper22 at 17:07 |

Nokia has caused a stir by performing, in the words of one security researcher, "man in the middle attacks" in order to compress data and speed up the loading of Web pages on some of its phones.

Nokia Asha phones send secure HTTPS data to Nokia servers, says security researcher.

The Finnish phone giant has since admitted that it decrypts secure data that passes through HTTPS connections -- including social networking accounts, online banking, email and other secure sessions -- in order to compress the data and speed up the loading of Web pages.

But, Nokia says that there is nothing to worry about.

Researcher Gaurang Pandya discovered that browser traffic from his Nokia (Series 40) "Asha" phone was being routed through Nokia's servers. This is no different to how Opera Mini works or even the BlackBerry browser, and remains popular in areas where the cell service is poor or in developing nations where cash doesn't grow on trees.

Nokia, however, goes one step further, the researcher says. A second post by Pandya, published this week, stated that Nokia was "man in the middle" attacking HTTPS traffic on its user's phones. In simple terms, HTTPS traffic was being routed through Nokia's servers, and could be accessed by Nokia in unencrypted form.

From the tests that were preformed, it is evident that Nokia is performing Man In The Middle Attack for sensitive HTTPS traffic originated from their phone and hence they do have access to clear text information which could include user credentials to various sites such as social networking, banking, credit card information or anything that is sensitive in nature.

He notes that whether be it "HTTP or HTTPS sites when browsed through the phone," Nokia has "complete information unencrypted (in clear text format) available to them for them to use or abuse."

Metasploit Penetration Testing Cookbook

Posted by jasper22 at 16:53 |

Введение

На сегодняшний день тестирование на проникновение (penetration testing) является одним из основных сценариев анализа защищенности. Пен-тестирование включает в себя полный анализ системы путем осуществления реальных тестов безопасности. Это помогает выявлять потенциально слабые места в основных компонентах системы: аппаратном или программном обеспечении.

Причина, которая делает тестирование на проникновение важным аспектом безопасности является то, что она помогает в определении угроз и слабых мест с точки зрения хакера. Лазейки могут быть использованы в режиме реального времени, чтобы выяснить воздействие уязвимости, а затем подходящим средством могут быть изучены для того, чтобы защитить систему от внешних атак и снижения факторов риска. Самым большим фактором, который определяет возможность проникновения — знание целевой системы. Есть несколько способов тестирования на проникновение:

метод черного ящика — осуществляется при отсутствии предварительной информации о цели. Для осуществления атаки, пен-тестеру приходится по крупицам собирать информацию о целевой системе
метод белого ящика - осуществляется, когда пен-тестер имеет достаточно полную информацию о цели

Промышленные специалисты определили некоторые ключевые шаги, которые необходимы практически во всех формах тестирования на проникновение, к ним относятся:

определение цели — сбор основной информации без физического соединения
выявление уязвимости - реализация различных методов обнаружения, таких как сканирование, удаленный вход (remote login) и сетевые сервисы, чтобы выяснить, какие службы и программное обеспечение, работают на целевой системе.
эксплуатация — использование уязвимостей (публичных или приватных) для атаки на службы, программы и т.п.
уровень доступа — атакующий может получить доступ на целевой системе после успешной атаки
отчет — подготовка отчета об уязвимости(ях) и меры противодействия

Этих шагов может показаться мало, но тестирование на проникновение систем высокого уровня (high-end system) с большим количеством служб может занять дни если не месяцы. Причина, по которой тестирование на проникновение выполняется так долго состоит в том, что применяется техника «проб и ошибок» (хотя к скрипт-кидингу это не относится в большей мере… там взял и применил). Уязвимости во многом зависят от настройки системы, поэтому мы не беремся утверждать, что рабочий эксплоит будет работать.

Глава 1. Metasploit — короткие советы

Настройка Metasploit в Ubuntu/Debian
Metasploit + BackTrack
Создание пен-тест лаборатории
Настройка Metasploit в BackTrack + SSH
Создание базы данных в Metasploit
Использование базы данных для хранения результатов тестирования
Анализ результатов из БД

Насколько плохим код должен быть?

Posted by jasper22 at 16:51 |

Эрик Липперт — ветеран Microsoft, проработавший в компании 16 лет и стоящий за разработкой VBScript, JScript и C#.

На прошлой неделе в комментариях к одной из статей разгорелся спор о роли низкоуровневой оптимизации в программировании, и я вспомнил относящуюся к этому статью Эрика. Она была написана в конце 2003, и хотя реалии с тех пор несколько изменились — принципы остались теми же самыми. Можете мысленно заменить ASP и VBScript на PHP, JavaScript, или на другой скриптовый язык по вашему вкусу.

Эту статью я уже пытался перевести в 2005, но русский текст тогда получился неуклюжий, так что этот перевод — новый и ранее не публиковался, в соответствии с требованиями НЛО. В Переводе блога Эрика Липперта этого текста тоже нет — наверное, для них он слишком стар.

Я уже много писал о быстродействии скриптов, но до сих пор я не высказывался по поводу того, что многие советы об их оптимизации я считаю как минимум бестолковыми, а то и откровенно вредными.

Например, за семь лет в Microsoft я получил десятки вопросов, аналогичных по своей сути этому, заданному в конце 1990-х:

У нас есть код на VBScript, и в одной часто вызываемой функции мы определяем оператором Dim несколько переменных, которые нигде в функции не используются. Не замедляется ли каждый вызов функции из-за объявления этих переменных?

Какой интересный вопрос! В компилируемом языке, таком как Си, объявление локальных переменных общим размером n байт всего лишь вычитает n из указателя стека при входе в функцию. Если n будет чуть больше или чуть меньше, затраты времени на вычитание никак не изменятся. Наверное, в VBScript точно так же? Оказалось, что нет! Вот что я написал автору вопроса:

Никчёмный анализ №1

Объявил переменную — получай переменную. Откуда VBScript может знать, не собирается ли функция выполнить что-то вроде

Function foo()

Dim bar

Execute("bar = 123")

Чтобы такой код выполнялся корректно, движок VBScript вынужден во время выполнения хранить список имён всех объявленных переменных. В результате объявление каждой лишней переменной отнимает время при каждом вызове функции.

Async Lambdas

Posted by jasper22 at 09:36 |

Today I learned something new and I love that!

I was looking at some code that looked like this:

try

{

await obj.GetSomeAsync();

Assert.True(false, "SomeException was not thrown");

}

catch (SomeException)

{

}

That’s odd. We’re using xUnit. Why not use the Assert.Throws method? So I tried with the following naïve code.

Assert.Throws<SomeException>(() => await obj.GetSomeAsync());

Well that didn’t work. I got the following helpful compiler error:

error CS4034: The 'await' operator can only be used within an async lambda expression. Consider marking this lambda expression with the 'async' modifier.

Oh, I never really thought about applying the async keyword to a lambda expression, but it makes total sense. So I tried this:

Assert.Throws<SomeException>(async () => await obj.GetSomeAsync());

Hey, that worked! I rushed off to tell the internets on Twitter.

But I made a big mistake. That only made the compiler happy. It doesn’t actually work. It turns out that Assert.Throws takes in an Action and thus that expression doesn’t return a Task to be awaited upon. Stephen Toub explains the issue in this helpful blog post, Potential pitfalls to avoid when passing around async lambdas.

A Smart Card Framework for .NET

Posted by jasper22 at 09:11 |

Introduction

The .NET Framework has been introduced in 2002, and the version 3.0 has just been released in November. So far, Microsoft hasn't included Smart Card classes in .NET, and if you want to develop a Smart Card aware application, you have to develop your own classes. Fortunately, it is much easier to reuse existing code in .NET than with Java. In Windows, if you need to use Smart Card, you just need to use the PC/SC API in your program. This API comes in C functions or COM objects that wrap the PC/SC functions. The .NET Framework offers two types of interoperabilities with the legacy code: the COM interoperbility, and the P/Invoke feature for native code interoperability.

Background

This article demonstrates how to use the interoperability features of .NET and use them to write a simple framework to use a Smart Card in your applications. A Smart Card is a small embedded device that receives commands through a card reader using the PC/SC Win32 API. If you want to use this API, you will need a Smart Card reader to use a Smart Card such as a SIM card.

A Simple Smart Card Framework

The SC framework I'm going to describe is composed of an interface to communicate with the Smart Card, a few classes to wrap the different parameters of a Smart Card command, and the implementation classes depending on the interop mode we are using.

The Smart Card interface provides a simple access to a Smart Card for .NET programs. We will see later how to implement this interface using both the interoperability techniques.

public interface ICard

{

string[] ListReaders();

void Connect(string Reader, SHARE ShareMode,

PROTOCOL PreferredProtocols);

void Disconnect(DISCONNECT Disposition);

APDUResponse Transmit(APDUCommand ApduCmd);

void BeginTransaction();

void EndTransaction(DISCONNECT Disposition);

}

The classes APDUCommand and APDUResponse are used to send the command and get the response from the card. SHARE, PROTOCOL, and DISCONNECT are constants used by PC/SC.

public class APDUCommand

{

public APDUCommand(byte bCla, byte bIns,

byte bP1, byte bP2, byte[] baData, byte bLe);

public void Update(APDUParam apduParam);

public override string ToString();

public byte Class;

public byte Ins;

public byte P1;

public byte P2;

public byte[] Data;

public byte Le;

}

public class APDUResponse

{

public APDUResponse(byte[] baData);

public byte[] Data;

public byte SW1;

public byte SW2;

public ushort Status;

public override string ToString();

}

INTRODUCING BDD

Posted by jasper22 at 16:39 | Wednesday, January 23, 2013

I had a problem. While using and teaching agile practices like test-driven development (TDD) on projects in different environments, I kept coming across the same confusion and misunderstandings. Programmers wanted to know where to start, what to test and what not to test, how much to test in one go, what to call their tests, and how to understand why a test fails.

The deeper I got into TDD, the more I felt that my own journey had been less of a wax-on, wax-off process of gradual mastery than a series of blind alleys. I remember thinking “If only someone had told me that!” far more often than I thought “Wow, a door has opened.” I decided it must be possible to present TDD in a way that gets straight to the good stuff and avoids all the pitfalls.

My response is behaviour-driven development (BDD). It has evolved out of established agile practices and is designed to make them more accessible and effective for teams new to agile software delivery. Over time, BDD has grown to encompass the wider picture of agile analysis and automated acceptance testing.

Test method names should be sentences

My first “Aha!” moment occurred as I was being shown a deceptively simple utility called agiledox, written by my colleague, Chris Stevenson. It takes a JUnit test class and prints out the method names as plain sentences, so a test case that looks like this:

public class CustomerLookupTest extends TestCase {

testFindsCustomerById() {

...

}

testFailsForDuplicateCustomers() {

...

}

...

}

renders something like this:

CustomerLookup

- finds customer by id

- fails for duplicate customers

- ...

The word “test” is stripped from both the class name and the method names, and the camel-case method name is converted into regular text. That’s all it does, but its effect is amazing.

Developers discovered it could do at least some of their documentation for them, so they started to write test methods that were real sentences. What’s more, they found that when they wrote the method name in the language of the business domain,the generated documents made sense to business users, analysts, and testers.

A simple sentence template keeps test methods focused

Then I came across the convention of starting test method names with the word “should.” This sentence template – The class should do something – means you can only define a test for the current class. This keeps you focused. If you find yourself writing a test whose name doesn’t fit this template, it suggests the behaviour may belong elsewhere.

How to get startup ideas

Posted by jasper22 at 15:43 |

November 2012

The way to get startup ideas is not to try to think of startup ideas. It's to look for problems, preferably problems you have yourself.

The very best startup ideas tend to have three things in common: they're something the founders themselves want, that they themselves can build, and that few others realize are worth doing. Microsoft, Apple, Yahoo, Google, and Facebook all began this way.

Problems

Why is it so important to work on a problem you have? Among other things, it ensures the problem really exists. It sounds obvious to say you should only work on problems that exist. And yet by far the most common mistake startups make is to solve problems no one has.

I made it myself. In 1995 I started a company to put art galleries online. But galleries didn't want to be online. It's not how the art business works. So why did I spend 6 months working on this stupid idea? Because I didn't pay attention to users. I invented a model of the world that didn't correspond to reality, and worked from that. I didn't notice my model was wrong until I tried to convince users to pay for what we'd built. Even then I took embarrassingly long to catch on. I was attached to my model of the world, and I'd spent a lot of time on the software. They had to want it!

Why do so many founders build things no one wants? Because they begin by trying to think of startup ideas. That m.o. is doubly dangerous: it doesn't merely yield few good ideas; it yields bad ideas that sound plausible enough to fool you into working on them.

At YC we call these "made-up" or "sitcom" startup ideas. Imagine one of the characters on a TV show was starting a startup. The writers would have to invent something for it to do. But coming up with good startup ideas is hard. It's not something you can do for the asking. So (unless they got amazingly lucky) the writers would come up with an idea that sounded plausible, but was actually bad.

For example, a social network for pet owners. It doesn't sound obviously mistaken. Millions of people have pets. Often they care a lot about their pets and spend a lot of money on them. Surely many of these people would like a site where they could talk to other pet owners. Not all of them perhaps, but if just 2 or 3 percent were regular visitors, you could have millions of users. You could serve them targeted offers, and maybe charge for premium features. [1]

The danger of an idea like this is that when you run it by your friends with pets, they don't say "I would never use this." They say "Yeah, maybe I could see using something like that." Even when the startup launches, it will sound plausible to a lot of people. They don't want to use it themselves, at least not right now, but they could imagine other people wanting it. Sum that reaction across the entire population, and you have zero users. [2]

Well

When a startup launches, there have to be at least some users who really need what they're making—not just people who could see themselves using it one day, but who want it urgently. Usually this initial group of users is small, for the simple reason that if there were something that large numbers of people urgently needed and that could be built with the amount of effort a startup usually puts into a version one, it would probably already exist. Which means you have to compromise on one dimension: you can either build something a large number of people want a small amount, or something a small number of people want a large amount. Choose the latter. Not all ideas of that type are good startup ideas, but nearly all good startup ideas are of that type.

Testing async Methods in C# 5

Posted by jasper22 at 14:02 |

Last week I promised that I’d write a blog post on using Assert.ThrowsException() to test async methods. Before I get to that, let’s go over some of the other issues that come up with testing async methods.

First, let’s look at a test to verify that an async method works correctly:

public async Task<string> GetMessage(string user)

{

return await Task.FromResult(string.Format(“Hello {0}”, user));

}

One test that works could be written like this:

[TestMethod]

public void SimplePathTest()

{

var worker = new Worker();

var answer = worker.GetMessage(“unit tests”).Result;

Assert.AreEqual(“Hello, unit tests”, answer);

}

I don’t like writing tests like that. Calling “.Result” on a task is a code smell. The test method blocks until the result is available. That’s probably OK in a test method (more on that in future blog posts), but I’m still concerned. Developers often copy code from unit tests into production code. (I’ve done it myself when I’m learning how a library works.) For that reason, I want my unit tests for follow the practices I would use in production code. That means I want to ‘await’ the result, not block for it.

Jasper22.NET