Saturday, December 22, 2018

Lies, Damn Lies, & Statistics

Lies, damn lies, and statistics.

So, I saw an article in the online edition of Sporting News this morning (http://www.sportingnews.com/us/mlb/news/average-mlb-salary-falls-for-only-fourth-time-in-50-years-first-time-since-200/y0pslbvbu9pf1u16kb4mubxt8) that said the average major league salary for 2018 was about 4.52 MILLION dollars. Since I'd not yet finished my first cup of coffee I stared at that number for a minute and then said "that doesn't make sense." My reasoning was this: there are somewhere over 900 MLB players (30 teams, 25 players each, plus all the players on the diabled list) and a few of them make enormous salaries - more than $20 million a year, while the minimum MLB salary is a paltry $545,000. By the way, the highest paid MLB player in 2018 is Mike Trout of the Anaheim Angels at a cool $33.25 million and the top 25 players in the league all make more than $22 million. So, $4.52 million as an indication of what the "average" MLB player makes seems way off the mark. And after some more digging I discovered I was correct. While the AVERAGE salary is $4.52 million, the MEDIAN salary is a meager $1.5 million. This is a much more accurate indicator of what MLB players actually are making. Now, nearly every article I found on my Google search reported the average salary figure and I had to dig around quite a bit to find the median. It strikes me that the stories that just reported the average salary were slanted towards the "wow MLB players make way too much money" opinion, and they emphasized the average deliberately (IMHO) because that number was so much higher than the median value.

Just goes to show you that you should always think about what you're reading in any media. (And just to say that I would have played for just one year at that average salary; just pick a position. Really. 😎).

Monday, September 10, 2018

New Book!

Springer-Verlag has just published my latest cryptology book History of Cryptography and Cryptanalysis: Codes, Ciphers and their Algorithms. If you read it, please let me know your opinion!

https://smile.amazon.com/History-Cryptography-Cryptanalysis-Algorithms-Computing/dp/3319904426/ref=tmm_hrd_swatch_0?_encoding=UTF8&qid=1536610952&sr=8-1


Monday, July 2, 2018

Billions and Billions and Billions of numbers...

So a week or so ago, someone got access to one of my credit card numbers and started buying things at the iTunes store. Capital One was on the job and immediately sent me text messages and emails asking "Is this you?" When I said "NO!" and submitted a fraud report they immediately canceled that card and got the charges rescinded and then within about 15 minutes they got me a new credit card number and said they'd be mailing me the new card (it arrived about 3 days later).  This is all very cool. It was a bit inconvenient because this particular card was the one I used for all my Internet purchases (I use a different card for non-internet stuff), but eventually I got it all straightened out.

But then I started thinking about Capital One and how many new cards - and card numbers - they must generate in a single day. I also started wondering about how many credit card numbers there were and when they'd run out of numbers (like the IPv4 internet addresses - there are only 4 billion of them and we actually ran out several years ago, hence IPv6). Clearly the credit card companies don't re-use credit card numbers. Also clearly there are a finite supply of them. So eventually we'll run out right? Well, here's the math...

All the credit cards I have and know about use 16-digits for your credit card number, plus you also have a security code of either 3 or 4 digits. The security code doesn't enter into this computation, though. So with 16 digits that is 10^16 possible credit card numbers - 10,000,000,000,000,000 possibilities. Also remember that there are about 7 billion people on the planet.

Now if all the credit card companies give out 1 million (10^6) new credit cards a day, that's 365 million a year. It will take about 3 years to get to a billion numbers (10^9), and 3000 years to get to a trillion (10^12), and 3 million years to get to a quadrillion (10^15). So I think we're good for a while.

But, a million numbers a day divided up between 7 billion people doesn't sound like very many. What if the credit card companies gave out 1 billion (10^9) new credit card numbers a day? Well, doing the same math we get 365 billion numbers per year. I will take 3 years to get to a trillion numbers (10^12) and 3000 years to get to a quadrillion (10^15) and 30,000 years to get to 10^16. So even if every person on the planet gets a new credit card every week we're still good for about 30,000 years. I'm happy.

But wait. There are other numbers. Your Social Security Number is only 9 digits long, which means there are only a maximum of 1 billion (10^9) possible numbers. There are currently about 325 million Americans with Social Security numbers, not counting all the dead people who had SSNs. So aren't we going to be running out of SSNs real soon now?

It turns out yes, and no. There's a great answer to this question on Quora that says that we've only used about 480 million of the billion possible numbers (there are actually fewer because some are excluded, like 000-00-0000 and 999-99-9999) and we're using them up at the rate of about 5.5 million a year. At that rate we'll use them all up in about 95 more years and we'll then have to either increase the length of your SSN, add a letter, or start reusing dead people's numbers. In 95 years I'd be over 150 years old, so I don't think I have to worry.

Friday, June 29, 2018

Furor over diversity in CS Higher Ed

A few days ago Stuart Reges, a CS prof at the University of Washington wrote an opinion piece at Quillette.com on "Why Women Don't Code". It's started quite the furor in the CS higher education community and garnered more than 325 comments and not all of them nice or civil. I won't even talk about the stuff that's come out on Twitter. There have been some good, well reasoned and reasonable responses. Here's a good response to Reges' article from Scott Jaschik at Inside Higher Education titled Furor on Claim Women's Choices Create Gender Gap in Comp Sci 

The best comment here is the first one after the article (by "Helen") that tries to be also reasonable and is also well thought out. After that first comment most of the men responding in the rest of the comments devolve into defensive drivel almost immediately...

While I think that Reges has a point about women making a choice not to pursue CS and I like his "equality" vs "equity" models, his argument is much too narrow and he doesn't consider many other contributing factors besides the intro courses and the welcoming attitude of the department that will go into those choices. But it's a good discussion to have. I also don't agree with Reges' defense of Damore (the fired and allegedly misogynistic Google engineer) since if I remember correctly, Damore's argument was that women weren't biologically capable of coding as well as men. That's just stupid and merits no defense.

These articles and the discussion they have started are worth having - as long as we can all remain civil.

Wednesday, June 13, 2018

Quitting my Ph.D. program

I've just retired from a full-time gig as a tenured professor of computer science. It was my dream job and I loved it. I taught in two different places for a total of 22 years. I also worked in the computer industry for a bit over 16 years, ending up as the VP of development for a startup software company. And I did it all without a Ph.D.

I've never really talked about the whole story of how and why I never finished my Ph.D. and I didn't really need to because I was lucky and found places that would hire me without one. But recently one of my former students has also quit her Ph.D. program and has just written an excellent essay on how that all worked for her. (her essay is here) So I thought I'd go through what I experienced when I quit my program.

In a sense, I quit two Ph.D. programs. I started grad school at Syracuse University as a Ph.D. student in mathematics (I have a B.A. in liberal arts with a major in math.) After my first semester of grad school I decided that I was not cut out to be a mathematician (that would be another long post) and I switched from mathematics to computer science. A year and a half later I got my M.S. in CS and high-tailed it out of upstate New York and went to work.

My problem was that, while I liked the software development jobs I got, what I really wanted to do was teach undergraduates. I was a math TA in grad school and really loved being in the classroom and interacting with students. Working in industry I missed that interaction. Really, my passion was teaching. After a year or so I started picking up adjunct teaching positions. I'd teach Introduction to CS, Data Structures, and the odd programming language class. That certainly helped and got me back in the classroom, but wow was I exhausted. Working full time in software development and then teaching for 3-4 hours in the evening, plus creating lectures, grading, etc. was a lot, even in your 20s.

So after a couple of years I decided to go back to grad school, finish the Ph.D. and get a teaching job at an undergraduate institution. This time I was at Rice University. At that time - 1978 or so - Rice didn't have a CS department. CS was split between the Electrical Engineering department (for the hardware bits) and Mathematical Sciences (for the theoretical bits). Since I was more interested in hardware and operating systems at that time I ended up in the EE department. Over the course of four years I finished all my coursework, passed my qualifying exams, and started research in networking algorithms (specifically using max-flow-min-cut algorithms to help design distributed operating system software). This should all have ended with me finishing the research, getting a publication or two, writing a dissertation, and then making my way into academia. Or not. As happens with many plans, life decided to interrupt. I went through a divorce. I was the first doctoral student for my Ph.D. advisor and it turned out he wasn't very good at it - and neither was I. It also didn't help that because I'd already been out in industry and in grad school once, that I was actually a year older than my adviser. I struggled with some of the mathematics we were using. And then just to make things more difficult a paper was published that proved that a similar research problem to the one we were working on was NP-complete, which kind of threw a wrench into our solutions. Finally, I frankly didn't really know how to do research at that time and I wasn't very good at it. So I got Rice to give me a second Master's degree, this time in EE, and I quit.

I packed it in and headed back to industry. For a long time I felt like a failure, even though I subsequently had a very successful career as a developer and a manager. I was convinced that I was merely a dilettante and was incapable of finishing any difficult intellectual task. This despite the fact that I was writing complex operating system software pretty much every day.

 The hardest part was that I really wanted to teach and for several years I thought that was just out the window. But, I finally started feeling better about myself. I decided that while failures on my part might be the reason for leaving my Ph.D. program, that in the end it was better for my life going forward. I still had many things to learn to get to the level that I supposed I needed to be at to do research and teach. It helped that early on in this process I got married to my wonderful wife of now 32 years and we had a son. I found a way to do some publishing while I was still working in industry, and I went back to teaching as an adjunct. Then I had the incredible luck to be fired from my last job in industry. Really. In my startup job my boss (who was a psychologist, not a tech person) didn't like some of the software project management innovations I was adding to the development process, so she fired me. That company is now barely alive, but there's no shadenfreude here; well, not much.  Getting fired was the best thing that happened to me.

Those things eventually gave me the confidence to apply for the teaching job of my dreams at Knox College. It also helped that at that time (2001) Knox had a terrific Dean who was willing to take a chance on a 48-year-old guy with two master's degrees to teach CS full-time in a struggling program. Sixteen years later I retired as a tenured full professor in an endowed chair and a thriving department with really terrific colleagues.

 I guess my point is that just because you've decided to walk away from one thing, doesn't mean you shouldn't walk straight into something just as good, or that with a little patience you'll end up where you should be. Quitting a Ph.D. program (or anything really) isn't about failure. It's about choosing what is best for you and opening up the opportunity to try something different and be a better you. It may take a while but you can surely get there.

Saturday, June 9, 2018

NSA Motivational Posters

I came across a new document on the web site governmentattic.org. It's in response to a two-year old FOIA request for motivational posters that the NSA used to put up. The request was specifically for posters from the 1950s and 1960s. The request was finally granted and the NSA shipped off a pdf to governmentattic with about 135 different posters in it.

They are hysterical and very reminiscent of the attitudes that were pretty generally in place during the height of the Cold War in the 1950s and 1960s. Here are a couple of the posters:


I just love the John Travolta one!  If you want to see all the posters they are at http://www.governmentattic.org/28docs/NSAsecurityPosters_1950s-60s.pdf

Enjoy!

Sunday, June 3, 2018

Hobbies and age...

Related to my previous post about hobbies, Vir Das at the Knox College Commencement today said (approximately), "You're never too old to learn a new hobby. You're just too old to do it well." I like that; it ties in well with my idea of being a dilettante as you grow older.

Wednesday, May 30, 2018

What to do in retirement? Part 2: Hobbies?

I've never really had a hobby as an adult. When I was young I used to collect stamps, mostly European and many of them pre-World War II, and I used to camp and hike (I was a Boy Scout for a while) and play pickup games of baseball and football, go swimming in the summer and sledding in the winter. And of course, I was a pretty voracious reader - but I don't count reading as a hobby, it's just something one must do.

But ever since I got to grad school and then went out into the working world, I've never really had a hobby. Life tended to keep me busy enough. It's not that I haven't tried. I took a class in woodcarving once and enjoyed it. I learned about different types of knives and chisels, I learned how to sharpen blades. But after carving a few small things (including a couple of very strange looking Santa Claus') I just kind of fell away from it.

Another time I bought a couple of books on furniture making and proceeded to make a couple of tables and bookcases. I enjoyed the mental and physical effort, the design problem, and had quite a bit of satisfaction when the pieces were done, however imperfectly. But again, after a few starts I kind of let things lapse.

Part of my problem, I think, is that in order to become good at a hobby that involves actually making something you've got to be very persistent. You've got to keep working at it, making new things and learning from each of your mistakes. It takes a long time to become good at your hobby. And I've never been that interested in the things I've taken up. Like a dilettante, I've always wanted to learn a little about a lot of things, but most of the time a little is quite enough. Also, I have a number of things that keep me busy around the house, yard work, small maintenance jobs, reading, movies I've put off watching.

So maybe taking up a new hobby isn't in the cards.

Or, maybe I already have a hobby.

If one thinks of a hobby as "an activity done regularly in one's leisure time for pleasure" then I've already got one - writing. (This isn't my idea, BTW. My wife said this very thing when I was whining about not having a hobby.)

I do research for my books, which I find enjoyable. Lots of times the research is done at my computer, but I occasionally have to travel to find materials. I get to correspond with others who do the same kind of research (these days it's in the history of cryptology). I like the process of organizing my research and turning it into prose that others might read. I even like the process of laying out and editing my work in preparation for sending it off to a publisher (or publishing it myself as I've done once). Because I don't depend on my writing as a steady source of income, nearly all of my projects can proceed at whatever pace I find comfortable. And I can put them down for a while to do something else, or go on a trip, or whatever, and then pick them up again when I want to. If that's not a definition of a hobby, I'm not sure what is.

So I think I already have a hobby, one that will provide many, many hours of pleasure over the coming years. So now I can stop worrying about finding a new one.

Sunday, May 27, 2018

What to do in retirement? Part 1.

I retired from Knox College last September. I'd worked in the computer software industry for a couple of decades, and for the last 16 years I'd been a professor of computer science.  I'd also done research in computer science education, software development, computer security, and the history of cryptology. I'd written 4 books and had lots of great students and colleagues (and some not so great, but that happens).

Since I've retired I've finished two more books, one on software development and one on the history of cryptology. I'd been working on them for quite a while and the book contracts just happened to be nailed down during my last year of teaching. When I was a full-time faculty member I could never get much research or writing done during the school year because at a small college you've got a ton of things to do during the year, none of which involve your own agenda. Realistically the only time I could do research and writing was during winter break and during the summer. Knox didn't have summer school so I had a leisurely (!) three months to work on my stuff.

As of last June, I'm not teaching and so I've got lots more time to work on research and writing. For the last 9 months I've been mostly head down finishing the two books. They are both now in the can; one has been published (see here) and I expect the galleys of the second in another week or so (see here).

So now I am looking at the prospect of doing....what? At the moment I've got no real deadlines. I have a book review to do and an abstract for a book chapter, but they are both sufficiently far in the future that I have no problem procrastinating. I have several other longer-term research projects I've planned to work on, but they can also safely be put off for a while. So I'm discovering that I'm a really good procrastinator. Without a real daily schedule, I find myself wandering around the house, doing chores, but not engaging intellectually. I've got a long, long list of books to read, and I really should post to this blog more often, but for some reason that all seems so hard to do at present.

Sorry for the whine. I'm actually in a really lucky position. Lots of potential projects. I still have lots of research and writing ideas. I have lots of chores and projects to do around the house. My wife and I have lots of travel plans and we love spending more time together. I'm healthy. It's just that I've realized that I need to be much more intentional about working to a new schedule, figuring out what each day will be like. I also need to learn to relax. It turns out I'm really bad at that. No time like the present. I think I'll go read a book.

Tuesday, May 15, 2018

Wednesday, May 9, 2018

Lets Fix the Electoral College and Gerrymandering, shall we?

Ideally, because they are supposed to represent all the people in the country and not the states, the President and Vice President should be elected by the entire country using direct election by popular vote. That's not going to happen anytime soon because the smaller states have too much clout in the current Electoral College setup to vote in favor of repealing it and replacing it with direct election.

So, instead, lets fix the Electoral College AND Gerrymandering

Here's an idea on how to at least partially fix the Electoral College (to help insure that the person who does win the popular vote does indeed also win the EC). And to make things better, I'll propose a second idea on how to fix gerrymandering.

(BTW, there's another proposal to fix the Electoral College problem without a constitutional amendment. It's called the National Popular Vote (https://www.nationalpopularvote.com/) and it requires states to pledge all their electoral votes to the candidate who wins the national popular vote. Great idea.)

SIZE OF THE HOUSE OF REPRESENTATIVES

First, the idea for the Electoral College. First of all, forget about ever amending the Constitution to eliminate the Electoral College and going for the direct election of the President. A Constitutional amendment needs a 2/3 majority in both houses of Congress, AND 3/4 of the states must approve of it. The smaller population states, particularly in the West, will NEVER go along with eliminating the EC. Along with the Senate, it's the thing that gives them clout far beyond what you'd expect from their population. Just forget about it.

Instead, what we need to do is to INCREASE THE SIZE OF THE HOUSE OF REPRESENTATIVES. The number of representatives in the House is NOT mandated by the Constitution, except to say in Section 2 of the 14th Amendment:
"Representatives shall be apportioned among the several States according to their respective numbers, counting the whole number of persons in each State, excluding Indians not taxed."

and in Article 1, Section 2, clause 3:
"The Number of Representatives shall not exceed one for every thirty Thousand, but each State shall have at least one Representative..."

The Permanent Apportionment Act of 1929 (https://en.wikipedia.org/wiki/Reapportionment_Act_of_1929) clarified when and how the size of the House of Representatives will be set. It set the number of representatives at 435 (which is where it had been since after the 1913 reapportionment as a result of the 1910 Census). (See also, Kromkowski & Kromkowski, "Why 435? A Question of Political Arithmetic", Polity, vol. 24, #1, Fall, 1991. http://www.thirty-thousand.org/documents/Kromkowski_Fall1991.pdf)

So it's been OVER 100 years since the size of the House of Representatives has changed. (https://en.wikipedia.org/wiki/United_States_congressional_apportionment)

This has lead to some anomolies in how many people are represented by a single representative in each of the states. Wyoming, with the smallest state population of just over 585,000 gets a single representative. Montana, with just over 1 million people, also gets just one representative. The most populous state, California, with nearly 36 million people, has 53 representatives, or one for about every 742,000 people. So the number of people each representative actually represents varies wildly across state boundaries.

If we increase the number of members of the House, then these anomolies can be evened out. This has been tried before (see Clemons vs Department of Commerce, 2009, https://en.wikipedia.org/wiki/Clemons_v._Department_of_Commerce), but not by trying an act of Congress.

If Congress enacts the so-called Wyoming rule (https://en.wikipedia.org/wiki/Wyoming_Rule) whereby each representative will represent a population roughly equal to the population of the smallest state, then the number of representatives will be increased from 435 to 537. California, Texas, and Florida will see the largest increases in representation (12, 10, and 7 seats respectively). Wyoming will still have a single representative.

This reapportionment will have the effect of giving voters in urban areas the clout that they do not have under the current numbers. It will also increase the number of Electoral votes to 637, with 319 needed to win the presidency.

(Another way to do this is to fix the number of people that a representative must represent, say at 600,000. Depending on how you count the remainders of state populations after you divide by 600,000 you'll get either 527 members of the House (if you use the floor function) or 576 (if you use the ceiling function). The effect will be the same as the current idea.)

GERRYMANDERING

Second, fixing gerrymandering (https://en.wikipedia.org/wiki/Gerrymandering and https://en.wikipedia.org/wiki/Gerrymandering_in_the_United_States). Gerrymandering occurs because in nearly all the states, the party that holds the majority of seats in the state legislature is the party that draws the boundaries for re-districting Congressional districts after each decennial census. This practice has gotten worse and worse over the decades, to the point where in several states, Wisconsin, Maryland, North Carolina, and Pennsylvania being the worst examples, the party that wins the majority of votes in the general election wins less than one-third of the seats in the legislature or in Congress. Here's a way to fix that.

Proposal: After each census, each state creates a committee with an equal number of Republicans and Democrats (sorry independents, you're out of luck here; if you want to play, declare a party affiliation; or how about a party gets to play if it polls more than 25% of the vote in the last general election?). If this is a state that only gets one Congressional representative, then we're done. For all other states, the party that won the most votes in the last general election goes first. That party draws one (1) Congressional district. Naturally, they'll draw one that favors their party. Then the other party gets a turn and they also draw one (1) Congressional district. The parties then alternate until all the districts for that state are drawn. All districts must be of approximately the same population (if the Wyoming rule is being used, then that population is the size of the smallest U.S. state) and they must be compact and contiguous - no snakes, or anchors, etc. The chief justice of the state's supreme court is the arbiter of whether the district is compact. It's likely that the last district will have either more or less population than the smallest state. There can be simple rules that will take care of this case.

In the end, there will be some safe Democratic districts and some safe Republican districts, but because of the alternation, there should be a number of competitive districts as well, almost certainly more competitive districts than there are today. Because these districts will be in place for at least a decade, it will be to both party's advantage to try to create a large number of closely divided districts.

With these two rules in place, the House of Representatives will more closely represent the majority of voters in their respective states and overall the majority of voters in the country. The Electoral College will also more closely represent the will of the majority of voters in a state; and in particular, the larger states will not have their votes in the Electoral College diluted by smaller states.

For other proposals to eliminate gerrymandering see
https://en.wikipedia.org/wiki/Open_Our_Democracy_Act
https://en.wikipedia.org/wiki/Gerrymandering#Changes_to_achieve_competitive_elections
http://www.rangevoting.org/
https://www.nytimes.com/2017/08/29/magazine/the-new-front-in-the-gerrymandering-wars-democracy-vs-math.html?mcubz=3

See also this article that illustrates how hard this will be to change:
https://www.nytimes.com/1992/04/01/us/supreme-court-upholds-method-used-in-apportionment-of-house.html