Database Design Class, Part II

Posted: January 24, 2012 in Graduate School

Noticed a couple of people missing from our second class. With these short 9-week long courses, missing a class would be tough. I wonder if they’ll be back this week. Anyway, my second class went a little deeper into SQL by adding nested queries, more complex joins, aggregate functions and grouping. Again, not much new for me there, but it will all be helpful for the next course, which focuses on implementation.

We got into E-R Diagrams this week and briefly touched on enhanced E-R diagrams. I found this part to be interesting because I’ve never really done E-R diagrams in the real world. It’s good to learn more about the conceptual side of things and to gain more discipline in thinking things through. Looking back, that’s one thing that has caused me problems in the past when I’ve actually had to design a database model.

For our class project, we’re working through a detailed set of data requirements this week. Although it’s mostly an individual project, we broke into groups of 3 for the purposes of having another set of eyes to review our project work before we submit our weekly assignment. That will help get some other ideas and thoughts about our projects, which is more like it would be on the job. We just don’t have the gazillion formal documents that have to go through various levels of committees and executive approval.

Next week we’ll touch on relational algebra and relational calculus (lets see how much math I’ve forgotten over the years). Yikes.

Database Design Class, Part I

Posted: January 16, 2012 in Graduate School

Rather than a traditional semester, the classes I am taking are divided into two terms per semester. The advantage is that I can take six credit hours per semester, but only have to concentrate on one class at a time. I may have to double up on classes a couple of times to stay on track though. Some classes are only offered every other year. At any rate, the class structure works out quite nicely in this fashion. Each term is 9 weeks in length and classes meet one night per week for 4 hours.

One thing I found interesting was the varied background of the other students in my class. A couple of my classmates have a similar amount of experience to my own. A few others have no exposure to databases whatsoever. In the middle are those who either have a small amount of experience with databases, or studied them in their undergraduate programs. I was actually rather surprised at how many of my classmates had an undergraduate degree in something other than Computer Science. Several had majored in Computer Engineering or Computer Electronics, one was an MBA student, and another was an accountant. I’m glad I have prior experience working with databases. It’s making it much easier to grasp the underlying theory and I expect that I will be in an even better position with the practical application exercises. I have to say that I admire those who will succeed in this class with little or no background in databases.

As for the class content, we began with a discussion of what a database is, what a DBMS is and other general terminology needed for understanding the upcoming discourse on theory. We then went over Codd’s Relational Model and ended with an introductory discussion of the SQL language.

I am fortunate to have a subscription to O’Reilly’s Safari online book service. On there, I found a series of video lectures by C.J. Date on the Relational Model (specifically, his master class on SQL and Relational Theory). I found this to be especially helpful as supplemental material. C.J. Date worked with Ted Codd and was one of the inventors of the Relational Theory, so you can hear about it straight from the source. He’s very strict in his interpretation and quick to point out where many others are erroneous. Good stuff – this level of detail is why I’m going to graduate school.

For next week we have a small homework assignment, some reading and need to provide an initial problem domain for our project. This particular class has a corresponding implementation class in the following term. For this class, we have a project to provide a database design for a problem domain of our choosing. For the next class, we will implement our design. I have elected to design a database for an online banking website. I wanted to choose something that I already have at least some familiarity with. Banking systems can of course be quite large and complex, so I am limiting the scope, but it should be sufficiently challenging and I look forward to working on the project. We end the class in our last week by turning in a bound presentation on our database design, along with a presentation to the class.

I’ve been thinking of ways to get back to blogging regularly. The main issues for me are time and topics of discussion. I believe I’ve found an answer. Although I thoroughly enjoy functional programming topics, I will veer away from that subject and move towards writing about my experiences in graduate school.

I just started my first class tonight in pursuit of an M.S. in Computer Science. One of the things I wondered about before starting school is just what to expect from graduate school. What are the classes like? What will I learn and how does that compare to my real world experience? What sort of projects will I be doing? What sort of background will my classmates have? These are the questions that are difficult to find answers for. It’s easy to find a curriculum and description of required classes, but that doesn’t tell you what it’s really like. So I thought it would be interesting to blog about my experiences and what I’m learning about. That’ll give me a solid 2 1/2 years of blog fodder in addition to providing an interesting way of reviewing and learning the material. In the end, I’d like to have something unique that others considering graduate school can look through to see what they can expect from a computer science graduate program.

I’m attending Webster University in St. Louis, Missouri. I will refrain from specific references to any persons or instructors, but rather concentrate on discussing the classes, course material, and overall experience. My intention isn’t to provide a review or rating of the program, but rather to provide information on what I’m learning and doing in class.

For Spring 2012, I am starting off with Distributed Database Design. In my next post, I’ll talk about my first class and what my next few blog posts will focus on.

The first thing I struggled with on my journey to learn about functional programming was where to begin. Looking back, I would have approached it a little differently, but hindsight is 20/20, right? My mistake was in trying to learn F# first. I figured that since I’m a .NET developer, I should start off with something familiar. Don’t get me wrong – F# is a very powerful language, but it’s not the best place to start when learning functional programming. I started working through a couple of introductory F# books and quickly found myself lost in learning the language syntax rather than understanding the underlying concepts.

That being said, you still need to work with code while trying to grasp the basic concepts of functional programming, so I would recommend starting off with Learn You a Haskell For Great Good!. Yeah, you’re probably a .NET developer and you’re asking, “Why should I read a book on Haskell?” Well, because this is not just a Haskell book, but an excellent introduction to functional programming concepts. I’ll come right out and say it’s one of the best programming books I’ve read. It’s all about the way the author explains things. It’s written in a gentle, concise, and light-hearted manner that keeps the overall flow running smoothly. While I did have to re-read a couple of things because I needed to walk away in the middle of a chapter, I was pretty much able to get a grasp on each concept on the first pass. There are way too many technical books out there that are full of good information, but can be so dry and technical, that I either lose interest at some point, or have to re-read several sections. This book is one of the few that has kept my interest throughout. Once you get Haskell set up on your PC (forthcoming blog post), just open up your favorite text editor, a command prompt, and then type in the code snippets as you read through the book. One of the cool things about functional programming is that it lends itself to running bits of code in a REPL environment. And hey, it’s good to work with something outside of Visual Studio now and then. Not only is it good exercise for your brain, it makes you appreciate how good we’ve got it now (back in my college days, I remember the pain of typing code into a mainframe, saying a prayer, and hoping it worked…but I digress).

Lambda Fever

Posted: June 9, 2011 in Functional Programming

It’s been awhile since I’ve blogged and it’s time to get back to it. In the not so distant past, I found myself wasting way too much time trying to absorb various technologies du jour, only to find that I really didn’t understand enough useful information to really help me with my day-to day development work. For us developers, it’s hard to decide what to focus on. There’s just too much out there to be a generalist. Yet, technology changes so rapidly that choosing to focus on one platform can leave you high and dry at any given moment. What’s hot today can be obsolete a year from now. If you’ve seen the Windows 8 demos, then you have to feel a tinge of pain for the folks who’ve invested their heart and soul in Silverlight and WPF. Microsoft kind of reminds me of one of my old girlfriends who told me “I think I’m breaking up with you but I don’t know. By the way, I’m dating someone else.”

So, I’ve taken a turn towards gaining a much deeper understanding of what’s going on under the hood in the various aspects of my day-to-day .NET development duties. I’m not interested in getting super-technical about the intricacies of the .NET framework. I’ll defer to the real experts on that. I’m a mere mortal. Rather, I just want to present my thoughts and experiences as I follow this path. Maybe someone will find what I blog about to be useful. Maybe not. I’m finding that blogging is a good outlet for gathering my thoughts together and increasing my understanding of whatever I’m blogging about.

So, like many of us in developer land, my employer asks that we set “goals” for the year. Regardless of your thoughts on the usefulness of setting goals that may or may not be looked at by the powers that be, I prefer to put some thought into it and take it seriously – for my own personal development if nothing else. This year, one goal that I chose was to gain a better understanding of functional programming (the tangible part being that I will do a presentation on it for my co-workers). I’ve been at it for a few months now, so I’ve made a fair amount of progress thus far.

I’ve got to say, it’s been awhile since I’ve been this excited about programming. I’ve thoroughly enjoyed my new quest into lambda land. It’s always good to work outside of the box and get those little gray cells fired up. Anyone who’s paid any attention at all to the evolution of the .NET framework can see the influence that functional programming has had on it from the introduction of generics, to LINQ and beyond. I believe all .NET developers should at least grasp the basics of functional programming. Trying to wield your .NET-fu, wrestling with generics, slinging around LINQ queries and delving into concurrency and multithreading issues without understanding what’s going on under the hood is like trying to learn an advanced belt kata when you’re a white belt. Now, I’m not saying that everyone needs to start writing F# applications, but you can’t deny the influence that functional programming has had (and will continue to have) on many of the features that .NET developers use daily.

And so begins the focus of the next several blog entries that I have planned. It took awhile before I got comfortable enough to even attempt to blog about my journey into functional programming. I will tell you right now that it’s not easy to grasp. At least not for me. You will absolutely have to think about programming in a different way. And hey, that’s a good thing.

Most of us have been doing imperative programming (I will explain this later). You may have heard of some scary things like Recursion (or even better, Tail Recursion), Currying, Monads, Functors, Higher-Order Functions, Partial Function Application, Immutability, Map-Reduce, Lambda Calculus, Category and Set Theory, blah, blah, blah. Just the associated terminology is intimidating enough to cause most developers to write functional programming off as cryptic, alpha-geek nonsense.

My intent is not to evangelize the use of functional programming, but rather to present to you the things that I’ve found useful in helping me to achieve that “a-ha” moment in seeing the beauty of functional programming and how it’s shaped the evolution of .NET development. I would also like to demonstrate why you should even care. You can easily take a big risk in pouring your heart and soul into any one particular technology platform, but it’s a pretty safe bet that understanding an underlying programming concept won’t leave you hanging. I can pretty much guarantee that taking the time to understand some basic functional programming concepts will make you a better developer.

I’ve gone Primal

Posted: April 19, 2010 in Other
Tags: ,

This post is a deviation from my technology focus, but I wanted something to keep me accountable for what I’m doing here. So if you want tech, skip this one.

A couple of weeks ago, I bought the Primal Blueprint by Mark Sisson. I’ve wasted too much time the past few years trying to change my lifestyle to be healthier (and having Zero success). I enjoyed success for a few years, back when I was regularly practicing Tae Kwon Do, but was unable to continue taking classes regularly since having kids and living too far away from my instructor’s school. Since then, I’ve been trying to figure out how to incorporate healthier living into my current lifestyle (at lot of working at home in front of a computer, 3 young kids, quick meals, family obligations, etc). This is the first thing I’ve run across that actually made sense to me and sounded realistic. I won’t bore anyone with details, you can read the book for that, or go to the website for Mark’s Daily Apple, but the basic gist of this is to live life the way our DNA is programmed for. That means trying to live and eat in a similar fashion to how hunter/gatherers lived thousands of years ago; before the advent of agriculture (and a largely grain-based diet) and more recently, heavily processed foods (junk food, fast food, food with preservatives, genetically-altered foods, etc). Again, you can read the book or check out the website for details, but I’ve recently started following the basic tenets of this lifestyle, other than a minor transgression at a Mongolian BBQ restaurant last week (I’ll make better choices going forward).

The results thus far, even in this short of a time period, have been impressive. I’ve been taking Prilosec for close to 3 years now. If I forgot to take it on any given day, it wasn’t more than a 1/2 day later that I was paying the price. My acid reflux had been increasingly problematic over the years until it got to the point where I was taking Prilosec daily. I stopped taking it after starting to follow the eating guidelines in the Primal Blueprint and my acid reflux is gone. I’m not sure what specifically exacerbated my heartburn – I had tried laying off of several things here and there in the past with no luck – but it hasn’t bothered me since I started eating better.

I’m not one for gimmicks, fad diets and quick-fit solutions. That’s one reason why I’ve been slow to get started because so many people out there advocate lifestyle changes with unrealistic philosophies. This is the first thing I’ve run across that truly makes sense and has some science behind it. We’ll see what happens down the road, but thus far, I’ve been very impressed with the results. Besides feeling better mentally about trying to eat well and cutting out the junk food, I already have improved energy and focus and have lost a couple of pounds. I’m eager to see where I’ll be in a few months.

update 

I can’t say enough about how much this can improve your life. It has definitely improved mine – not only physically, but psychologically as well. If you follow this plan and stay on it (no shortcuts here), you absolutely will lose weight and feel better. I’ll be honest and say that I don’t follow it 100% – there’s still room for improvement. But I’ve lost well over 60 pounds and kept it off for over a year thus far, and frankly it hasn’t been difficult at all. I still have a piece of cake when there’s a birthday, or a some fried chicken once in awhile. Don’t expect to be 100% strict on your diet – that’s unrealistic. Stay on the course 80% of the time and you’ll be fine. Once you’re on this plan, your body will actually handle the “bad” food better on the rare occasion that you “indulge.”

Medically speaking, the heartburn I used to have is still gone. I just had a comprehensive set of blood tests done last month. I’m happy to say that I’m either right where I should be, or better on most of the major health indicators (cholesterol, blood pressure, blood sugars, etc.). I had always been in the “at-risk” category on a lot of those before. Overall, I feel so much better that I did before I started. My focus now is to start taking the exercise more seriously. This lifestyle doesn’t advocate the “chronic cardio” that many people lose themselves in, but you do need to get out and do something to gain the full benefit. Eating well takes off around 70-80% of the dead weight. The exercise and other factors should take care of the rest.

In the words of Zeddicus Z’ul Zorander, “Bags! Nothing is ever easy!” (from the Sword of Truth series). That makes me think part of Zedd’s wizard training must have involved working with SharePoint, but I digress. For a project I’m working on, I’m using the WCF REST Service Kit to provide a syndication feed based on data contained inside of SharePoint. No, the OOTB RSS feed for a SharePoint list doesn’t cut the mustard for this one. I’m running MOSS 2007 on a Windows Server 2008 machine. I needed the WCF service to run with anonymous authentication. I wanted to call the WSS object model directly, rather than having to use the SharePoint web services. I’m still a novice when it comes to WCF, so some things that I learned along the way:

1. Calling the SharePoint object model from a web application can be tricky. One of the problems you’re likely to run into initially if you’re working on 64-bit machine, is this error:

“Could not load file or assembly ‘Microsoft.SharePoint.Search, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c’ or one of its dependencies.”

 

Most of what you’ll find online will say something about trying to run a 64-bit application in a 32-bit process, or my favorite, if you want to call the SharePoint object model from an application, you need to have SharePoint installed (don’t just copy over the SharePoint DLLs from the server to your dev machine). Ultimately, this article on MSDN is what I needed to see.

2. You may have a problem getting the WCF service set up on your server and get an error message like this:

“IIS specified authentication schemes ‘Basic, Anonymous’, but the binding only supports specification of exactly one authentication scheme. Valid authentication schemes are Digest, Negotiate, NTLM, Basic, or Anonymous. Change the IIS settings so that only a single authentication scheme is used.”

 

This article from Sahil Malik is a great explanation of what you need to do to get the WCF service set up on your server. This will help you avoid the error above. The key here is to host the WCF service in a web site that does not host a SharePoint application.

3. Once I had the WCF service set up in its own website, I was able to browse to the .svc file successfully, but when I tried to retrieve the syndication feed, I got the Internet Explorer page that says there’s a connection problem, displaying a “Diagnose Connection” button. At first I thought, “huh?”, and then saw that I was getting a 403-Forbidden error. Okay, so now I know it’s an authentication issue of some sort. I went into IIS and switched the web site to use Windows authentication. Now, when browsing to the syndication feed, after entering my credentials when prompted, everything worked. But I don’t want to be prompted for credentials. I reset the web site to use anonymous authentication.

So, when debugging the code, I was able to see that it failed when trying to make a call against the SharePoint object model. I tried many different things, but ultimately what worked, was to use the SPSecurity.RunWithElevatedPrivilege method to get a SPUserToken, and then create the SPSite object by passing in the user token. Once I did this, everything worked perfectly.

I thought I’d put this blog post together hoping it summarizes the common issues you may run across trying to create a WCF service that calls the SharePoint object model. I read many, many posts and articles online, and I didn’t get my answers from any single post, so I wanted to put together something to highlight the key ones that helped me out.

I started off with the Atom Feed WCF Service project template. This bit of code is what I used to retrieve data from SharePoint and create a List of SyndicationItem objects that can be used to provide a syndication feed:

List<SyndicationItem> items = new List<SyndicationItem>();

SPUserToken systemAccountUserToken = null;
SPSecurity.RunWithElevatedPrivileges(
    delegate()
        {
            using (SPSite site = new SPSite(Sites.AdminSite))
            {
                systemAccountUserToken = site.SystemAccount.UserToken;
            }
        });

using (var site = new SPSite(Sites.AdminSite, systemAccountUserToken))
{
    using (SPWeb web = site.OpenWeb())
    {
        IList<Announcement> announcements = _repository.GetAll(web);

        foreach (Announcement announcement in announcements)
        {
            var item = new SyndicationItem(
                announcement.Title,
                announcement.WebAnnouncementBody,
                new Uri(Links.AnnouncementLink),
                announcement.Id.ToString(),
                announcement.AnnouncementStartDate);

            items.Add(item);
        }
    }
}

This one caused me plenty of snarling and teeth gnashing yesterday. The problem surfaced when I wanted to do a quick test on some code, starting off with something like this – “SPSite site = new SPSite(url).” This line caused the following infamous exception to be thrown – “…FileNotFoundException: The Web application at http://moss2007/ could not be found. Verify that you have typed the URL correctly. If the URL should be serving existing content, the system administrator may need to add a request URL mapping to the intended application.” You’ll find plenty of posts for this exception online, covering various issues and resolutions (for example, see Paul Galvin’s post here, this one on StackOverflow, and this one in a Bamboo Solutions forum). Most of what I found online was similar to one of those items and none of the proposed resolutions fixed my problem.

I was running a 64-bit virtual machine with VS2008, Windows Server 2003 R2, and MOSS 2007 with SP2. I also use ReSharper’s test runner and TestDriven.Net. During my investigation, I went back to a SharePoint virtual machine that I know for a fact had several unit tests that ran perfectly fine when I used it in January ‘09. I ran a test there, and I saw the same issue. After scratching my head, because I knew these tests had run perfectly before, I noticed that I had opened the VM back in May and Windows update did its thing, so it wasn’t exactly as I’d left it in January. I also discovered that I encountered the same issue whether I ran code in a console application or through a unit test harness. I found another SharePoint virtual machine that I hadn’t touched since Dec ‘08 and it was indeed free of any updates since then. I found that the unit test harness and a console application ran code just fine with no errors. Hmmm. I went back to the virtual machine I first encountered this problem on, and restored a snapshot of it back when it had MOSS 2007 SP1. It still had Windows updates throughout 2009 on it, but not MOSS SP2. It still had the same issue. I was looking through the ULS logs and noticed some errors indicating an HRESULT of 0×8007002 with the text like – “There is no Web named "/sites/intranet/Lists/Templates/AllItems.aspx". An online search led me to this post, detailing the exact error I was seeing in the logs. One thing that stood out to me was a reply in the thread asking the poster if they had the latest cumulative updates installed. I thought to myself “what the heck,” so I went back to my virtual machine, restored it to MOSS 2007 SP1 again, then ran the WSS/MOSS SP2 executables, as well as the Oct ‘09 cumulative updates. Oddly enough, this resolved my issue for the most part. I really have no idea what changed, but I was able to run code from my test console application again. I then tried to run a test through ReSharper’s test harness, and it failed as it had before. However, I was able to run tests with TestDriven.Net just fine. So everything works now, except ReSharper’s test harness.

If I have some time I may try investigating this further, but I was able to resolve my issue well enough by installing SharePoint SP2 and the Oct ‘09 cumulative updates. I’m not sure if it has something to do with the issues others have had due to differences between 32-bit and 64-bit processes. My application was set to AnyCPU initially, and trying to set it straight to 64-bit made no difference. The posts focused on database permissions issues I saw online weren’t a factor for me, and neither was fiddling around with the identity the AppPool was running under.

Now, I hope this problem doesn’t bite me when I start playing with SP2010 (see this post on SharePointDevWiki).

I recently downloaded the Ethical Wall sample from Microsoft for a quick proof of concept. The application was written for OCS 2007 and is setup to run in VS2005. Since we need to run this against OCS 2007 R2, I needed to compile the application on my OCS 2007 R2 development machine in VS2008. The projects have a reference to the Managed server API platform (ServerAgent.dll). You’ll see the invalid reference to the ServerAgent.dll in the project references the first time you open up the solution. I couldn’t find the ServerAgent DLL when I brought up the references dialog, so after poking around online, I found that you’ll need to install the Microsoft Office Communications Server 2007 R2 SDK, then you can find the ServerAgent.dll inside of the bin directory where the files are installed. Hope this saves someone some time trying to resolve the reference to ServerAgent.dll when trying to compile this in VS2008.

It’s been a long month

Posted: November 21, 2009 in Productivity
Tags:

I’ve been busy this month finishing up one large project, picking up a project from someone else, and trying to squeeze in a third project. Hopefully I’ll get a chance to write a meaningful technical post soon. I was fortunate to have attended the SharePoint Conference last month and have been keeping busy trying to absorb new material in what little free time I’ve had lately. It’s always a balancing act trying to decide what to attempt to keep up with. I’ve worked on various .Net development projects over the years, but now is the time to concentrate on something and get more specialized. As much as I’d like to keep up with everything that interests me, it just isn’t possible.