Blake's Coding Blog

56 Days of Second Life

2012-01-05T14:17:00.001-06:00

Coming back from a long hiatus (wow, has it really been a year?), I went back to school earlier this week. My new classes have officially started, and boy are they monsters! The good news is this is the last stretch for me before graduation. The bad news is that these classes are going to eat up every ounce of my spare time.

One of the two classes I'm enrolled in now is an Emerging Technologies course. Along with a couple other assignments, we have to work on a project that demonstrates, well, emerging technologies! I've decided to do an 8-week-long project focusing on the social interactions that take place in Second Life. It was between that or a robotics project, and I don't feel confident enough with physical electronics to have my academic success ride upon it. At any rate, this project should be really interesting! If you'd like to follow it, I'm required to maintain a blog detailing my findings and analysis of what I observe. You can find it over here.

On a related note, expect an update on SDS pretty soon. It'll probably upset a lot of people, but it's high time I speak up about it. See you in a day or two!

SixDegreeSteam 1.6.6 "Pure Client" Beta Live

2011-01-22T14:12:00.002-06:00

Im very proud to announce the release of SixDegreeSteam version 1.6.6 ("Pure Client") available at the SDS site. This is a closed beta that anyone can download and use, but will only work with Steam profiles that are members of the SDS Steam group. This version is NOT meant to be representative of SDS, which is to say this is just a technical demo. It doesnt include all planned features, is expected to run pretty slowly (its dependant on your processor, memory, and internet speed), and will most likely not find relationships larger than 5 profiles long (at least not with the default user limit. Even if you up the limit, itll still take a really long time). So what DOES it do? Heres the list:

DOES find your relationship to closely related users (5 or less recommended)
DOES print found relationships in a list structure
DOES show the potential of the project, and why the dependence on the server is so important (seriously, do you really want to have to use a program this slow?)
DOES NOT generate links using groups or private profiles
DOES NOT have a graphical user interface (unless you count the command prompt as graphical)
DOES NOT print the relationships in a very user-friendly manner (yet...Im working on that)
DOES NOT find relationships unless at least one of the given profiles is a member of the SDS Steam group

This is really only meant to show the current standing of the technology and give a sense of why the project is taking so long to complete. Trying to get something that is inherently very slow to work fast enough to be usable is not an easy task. Just remember that this is only the start of a potentially great project. Please be patient.

In the meantime, I hope everyone who tries the beta understands that it isnt, nor is it meant to be, very good or user-friendly. There is a lot to still be done. If you have comments, please let me know. I love hearing from you all!

Thanks so very much for your gratious and humbling interest,
Blake (ROOT)

Drunken F00l Got Ripped, Son!

2010-07-07T18:30:00.002-05:00

So, one of the biggest names in the Team Fortress 2 world got banned by the Valve Anti-Cheat system recently. Drunken F00l's profile is now marked as VAC banned. Most likely, this was because he received one of the most coveted in-game items at the moment: The Golden Wrench. Only 100 copies of this item were to be given out randomly, and he received one supposedly via an exploit.

The funny thing is he also has one of every hat, which just doesn't happen. The focus now turns to whether he has always had the ability to get any in-game item he wanted. Either way, it's a hillarious consequence toward a classic, yet undeniably influential, douchebag.

F00l has been behind a bunch of different TF2 modifications, both good and spitting-in-the-face-of-Valve bad. TF2 Items, the idling programs created to unfairly gain in-game items, plugins like SourceOP, and multiple new game modes ~~such as Prop Hunt~~ were all his doing. (Edit: It turns out Prop Hunt was not an original concept by F00l. Instead, it was a clone of a SourceMod plugin created by Darkimmortal. Thanks to Darkimmortal and Geit for pointing this out)

We salute you, Valve! Keep doing what you're doing! And we hate you, F001! Just because you're one of the most recognized TF2 modders doesn't mean you can get away with everything!

Getting Past OnLive's LAN Requirement

2010-07-03T17:34:00.002-05:00

I was lucky enough to be part of the OnLive Founding Members promotion since I pre-ordered way back when the service first came around. If you were too, you might know about the current restriction in place on WiFi connections. Basically, it's a temporary means to limit the ways players can connect to the service in order to rule out connection reliability as a factor during these early stages when troubleshooting is inevitable. As far as I know, this is only meant to help cut down on the amount of calls-for-help from the technologically inapt. Imagine every noob using the service via a wireless connection flooding support forums, inboxes, and phones with problems pertaining to their network connection speed, which is totally not OnLive's problem/fault.

However, I don't think it's all that fair to limit those of us with prominent networking experience and wireless connections. So, I found a way past it. WARNING: Using this method of connecting to the OnLive service will most likely make you ineligible for support from OnLive until either wireless connections are supported or you actually connect through LAN. YOU HAVE BEEN WARNED!

The fix is very simple and can be done on Windows Vista or 7 (perhaps other versions as well, but I haven't been able to verify). Follow these steps:

Right click your network connection icon in the notification area, then choose Open Network and Sharing Center.
1. OR open your Control Panel, click the Network and Internet category, then Network and Sharing Center.
Click Change adapter settings in the left pane.
Select the wireless network connection you use to connect to the Internet and an unused local area connection (click one, hold CTRL, then click the other).
Right click one of the selected items and choose Bridge Connections. A Network Bridge item should be created.

Enjoy! The only real side effect I noticed about this procedure is that your network connection icon may display the disabled/unconnected icon even if you are fully connected to the network and Internet. If you don't mind that, this work-around is pretty solid. When OnLive adds WiFi support or if you decide you want your LAN connection back, just go back into the adapter settings and delete the network bridge item.

PS: Add me as a friend or spectate me sometime! My gamertag is Blakeo_x.

SDS Status Report: February 10

2010-03-01T17:18:00.000-06:00

The past month has been very dull for the SixDegreeSteam project, unfortunately. A lot of personal baggage has bullied progress to the bottom of the stack. However, with a new month comes a renewed working spirit. February wasnt completely without change for SDS, though. The general crawling was halted early in the month. The decision to do so was based on the fact that an accurate backbone has already been developed (the purpose of the general crawling) and that crawling on a relational chain basis, which is much more time- and resource-friendly, will likely replace a huge portion of the general crawling dataset upon client release. The SDS crawler has also reached a new version. SDS Local Server 1.6.1 reduces the residential memory to the SteamUser class, features a slightly modified database querying scheme to [fractionally] speed up present and future data acquisition, and utilizes a new community page wherein friend data is stored in XML format, speeding up crawling and reducing bandwidth consumption.

March is likely to be a very productive month for SixDegreeSteam. With any luck, the client will be ready for public use soon!

SDS Status Report: January 10

2010-01-31T20:04:00.000-06:00

A full month of crawling has passed, leaving the database pretty full and everyone involved with the project ecstatic to see everything coming together. A lot of progress has been made since the last report, so there is a lot to report on here. Strap in!

On January 3, this discussion took place:

January 3, 2010 chat between Blake "ROOT" and Harry "BlackSyte":

Blake: SDS is running a lot slower now that things are filling up, and the partitioning [of the database in an attempt to make things faster] didnt go over too well. My next attempt will be to do manual partitioning, but I dont want to do it right now because I just worked pretty hard to repair the damage the automatic partitioning did...It slowed things down to 1 process per minute.

With the current table loads and everything back to normal, its about 50 profile processes per minute. Its not THAT bad, but it isnt fast enough. Something exciting, though; SDS has reached the edge nodes. The very first Steam user has been crawled, and the very last Steam user (as of this writing) has been crawled. All that is left is everyone in between! But, it stands to prove that my theory might hold weight afterall. It shows that someone, thereby everyone, in the SDS Steam group is connected to both the minimum and maximum edges.

On January 18, SixDegreeSteam Local Server 1.5.87C (the current version) was implemented. It is thus far the most stable release with many improvements over the initial release from December. Family 1.5 brought in better error handling, a logging mechanism, parameter-based execution, a "prioritize child nodes" option for high-priority queue entries, and some programmatic fine-tuning. Release 1.5.87 introduced multi-threading for running multiple local servers (crawlers) consecutively and fixed some profile tracking issues that were previously irreproducible in production.

On January 21, the client algorithm was successfully executed. The interface is still a couple weeks out, but is coming quickly. Work on the interface was stalled after the 21st due to coursework and may continue to be stalled for 4 more weeks.

As of this writing, the database reports these statistics:

- Users crawled: 2,518,356
- Groups crawled: 748,140
- Profiles pending: 5,988,765
- Total discovered users: 7,791,431
- Total discovered groups: 748,141
- Average time between discovery and crawling: 9 days

Bonus: This is a discussion that took place on January 5. It doesnt do much to prove progress of the project, but does offer some interesting food for thought:

January 5, 2010 chat between Blake "ROOT", Dallas, and Harry "BlackSyte":

Blake: Its amazing how quickly the graph edges progressed.
Dallas: But the surface has only been barely scratched.
Blake: Exactly! I mean, we arent even half way through all the users, yet a streamlined backbone has emerged. That raises a concern...Perhaps there are more profiles with a betweenness centrality less than or equal to their degree centrality than I thought. Since a rigid, well-defined backbone has already formed so early in the project, yet there are relatively no user discoveries, it makes me think[...]

There are two possible conditions under which this would happen, guessing that the crawler itself is not at fault, and my confidence of that is fairly high as of 1.5C. First [possibility], there are a shitload of people who form "clicks" [or] small collections of friends who do not join groups or befriend "outsiders"...and by shitload, I mean over 80% of the entire demographic. Thats hard to believe, but not improbable.

The second case, which is even mmore unlikely but is very possible given a generational standpoint, is that there are sections of the entire demographic who are only friends with other members of the same section. So, you have 4 million users in segment A who are friends with other users in that same section, but none of them are friends with users from section B.Its incredibly unlikely, but its a valid portrayal of an existing graph theory called generational demography -- Newer users tend to be friends with other newer users, while older users tend to be friends with other older users, and never shall the two meet.

Heres another interesting observation Ive made. The queue timeframe is currently 9 days...Now, what that means is that there is a 9-day waiting period between discovery and crawling. Its a common occurrence in the crawler that a profile will become invalidated within those 9 days; A wildly common occurrence, in fact. People are deleting their profiles or changing their profile names way too often, somewhere between 1 to 9 days!
Harry: Does that hamper the crawling process?
Blake: In the first release, yes. The crawler would actually crash with a fatal error because it was expecting the profile to be there, but it wasnt. That would happen in the first couple days of launch before I patched it, which is pretty funny because that means the profiles were becoming invalidated within 1-3 days!

SixDegreeSteam: Synopsis

2010-01-14T16:32:00.001-06:00

I was curious as to a means that people like myself could come to an understanding on roughly how this program works... I understand what the goal of it is, but how do you intend to do this?...

- TopRaman

While I have explained a lot of SixDegreeSteam's abstract mechanics in previous articles, I feel they were not programmatically detailed enough to give a good picture of how exactly the program operates. So, Ill try to fill that gap here. There are three components to the project; Ill try to discuss all of them as simply and cleanly as possible.

WARNING: Previous articles have focused on previous versions of the project. Likewise, this article with focus on the current version of the project as of this writing (1.5C). I feel this current version is the most efficient, so it is likely to stick around for a long time. But, this is a disclaimer just in case the project does change again substantially.

The first and (thus far) most time-consuming component is the crawler. Without it, the project would not have a dataset. The crawler has the straight-forward job of collecting links to other profiles from one profile, saving (queueing) those links, then opening them back up sequentially to collect more links. The process continues until all links are collected and thereby all profiles are analyzed. This process of collecting, storing, reading, and repeating is called crawling. The specific crawling logic will not be described here for the sake of brevity, but all the details can be found in an earlier post titled SixDegreeSteam: Challenges. Program-wise, though, the crawler downloads the pages containing the links, extracts the links using a combination of Regular Expressions and XML parsing, and inserts them into a SQL database table called the crawler queue. The crawler also extracts some basic profile information, such as SteamID, profile name, and avatar, in a similar manner and inserts it all into another table called the user dataset (or group dataset). The complete list of datums stored per user is SteamID, last crawl time (to prevent recrawling a user too frequently), profile name, avatar, friends, and group memberships. The complete list of datums stored per group is similar: SteamID, last crawl time, group name, avatar, and members.

Given that the crawler does its job correctly, we are left with a database filled (and I mean FILLED) with information about the users and groups of the Steam Community. Oddly enough, the information is pretty inconsequential without a way to harness the datasets. So, our second component, aptly named Pathfinder, is arguably as important as the crawler. Pathfinder has the soul purpose of using the information available through the database to calculate a lowest-cost path between two given nodes. To do this, Ive opted for an object-oriented rendition of the breadth-first search graph theory algorithm. Once again, for the sake of brevity, I wont go into detail about the algorithm. If you are curious about it, follow the link or run a Google search. There are tons of articles that have covered it far better than I can. After the algorithm is applied, a list of users and groups used to reach profile B from profile A the quickest is displayed with links to each profile and avatars for recognition purposes.

The third and newest addition to the component set is a sort of network browser. Using a graphical, navigable web of the dataset, users will be able to quickly traverse the entire Steam Community social network, allowing them to get a better idea of just where they fit into it all. This component is simply an auxiliary to the project and is not a real focus. As such, it will be the last to be implemented and will only even enter the development picture after the crawler and Pathfinder are thoroughly completed.

As was said about Pathfinder, the information gathered by the crawler is pretty useless to the end user until a method of putting it to work is created. This explains why the crawler has been operational for nearly a month now, yet the site is still empty.

SDS Finished (For Real This Time!)

2009-12-27T21:20:00.000-06:00

The centerpiece of the project, the SixDegreeSteam crawler/server, has been thoroughly completed. Since the promised launch date of December 25, the crawler was experiencing memory issues that would cause it to die roughly 5 minutes after startup. Im very pleased to announce that version 1.5.77C of the crawler (yes, it seriously took that many revisions to get everything up to working conditions) is now in effect, and running smoothly!

Now I can switch over to working on the client software. More about that will be released later. Here are some statistics about the limited runtime the crawler got between its crashes.

- Uptime: 2 hours
- Users crawled: 1,961
- Groups crawled: 10,337
- Users/Groups pending: 408,376

A special thanks to everyone who has sent me emails, joined the Steam group, and otherwise completely overwhelmed me with their gracious comments! Its truly inspiring to see so many people in support of what I invisioned to be a small personal project. As always, feel free to email me with questions, comments, concerns, or funny lolcats. blake.oxx@gmail.com (remove the dot)

SixDegreeSteam Finished

2009-12-16T19:46:00.000-06:00

After about 3 months of development, SixDegreeSteam has reached the final steps in the launch process. The database has been set up, the crawler has been completed, and the folks at Valve have been sent "please let me borrow your bandwidth" cookies. Everything is pumped and primed. However, the project has not yet been released on the community. Why? Because Valve has not yet agreed. As soon as the green flag is given, the experiment will begin full-force.

As a side note, a previous article I posted about how SDS would acquire information is wrong. Actually working on the program and digging through the Steam Community site has enlightened me a little more about the crawling process. The article will be corrected as soon as I have time.

SixDegreeSteam Mailbag 2

2009-11-29T13:26:00.004-06:00

@SixDegreeSteam: Bump bump bump! Updates, please!

- Anonymous

Good timing! It just so happens there is yet another development in the SDS programming progress. I decided to set aside all of the existing code for further development and implementation at a later time, and start over yet again. In this new iteration, I intend to keep things simple with a focus on mechanics and practicality, not resources and scalability. This aids the project in two ways. First, interest in hosting slave servers has been very low. So, it seems like a waste to work so hard on a component that may not even be used (or necessary, for that matter). Second and most obviously, deployment has stalled, and stalled, and stalled some more. I am very dissatisfied with the current timeline for completion compared to the originally proposed timeline. Going back to basics will speed up development phenominally and consequentially lead to a much closer launch date.

Where will the project be accessible from?

- Anonymous

The brand-new subdomain on my brand-new server/site will be the point of deployment. While we are on the subject, I will also be moving my blogs to the new site once the general site is set up.

This article is a reply to an email. If you would like to ask a question or suggest a new article, email me. blake.oxx@gmail.com (remove the dot)

SixDegreeSteam: Programmatic

2009-10-24T17:30:00.000-05:00

First of all, thanks to everyone who have subscribed to my Devlog, sent me emails, or have otherwise expressed great interest in SixDegreeSteam. You all serve as my motivation, and I am very proud to be so highly regarded. Thank you all! Now, as promised, here is another SDS tech article.

From the conception of the project, SixDegreeSteam has changed fundamentally a couple times. The reason for this is because there are so many factors to consider that it is near impossible for me alone to take them all into effect. I've sat down and worked out the best approaches I can conceive, but after putting them all into working code, something new will show itself and justify a rethinking. The major issue here is that there are two conflicting, equally and critically important aspects to this project; bandwidth and memory usage.

Bandwidth and memory are quasi-mutually exclusive. In other words, you can not optimize one in a program that utilizes both technologies without negatively impacting the other. The reason for this is simple. In SixDegreeSteam, with the exception of static and very few dynamic data, anything that we transfer (bandwidth usage) is useless to the crawler and is therefore unnecessary to keep in memory (memory usage). Likewise, anything kept on the client side for a longer time, indefinately, or otherwise in a non-time-critical manner is not likely to be needed by the server, hence memory on the client is used in favor of bandwidth. While this trade-off does have its implementations, bandwidth is a much more precious commodity in the long run. Memory can be reclaimed by throwing away data, but bandwidth comes in limited amounts (for most of us) on a monthly basis. However, the more memory available to the program, the faster and larger the resulting dataset can be. I've had to rewrite the messaging protocol between the server and client once, the crawler module thrice, and the program base twice as a result of trying to find the best trade-off between the two technologies. I have a sick feeling there will be a few more rewrites to come, and many more after the project goes public.

The bandwidth-versus-memory conundrum has led to a change in the structure of the program as a whole. In previous articles and discussions, I spoke of a multi-threaded program layout consisting of a server and a client (slave threader), with the client managing "slave threads" responsible for the actual crawling operations. As of this writing, multi-threading is still a part of the server aspect in order to allow the server to communicate with multiple clients at once. But, the client has been changed from the slave threader approach to a single-thread, single-crawler, tree-based data collection scheme. The client starts by requesting a Steam profile to begin crawling from the server. Then, an object at the top of a tree data structure is created to represent the profile. The object then crawls the ID, friends, and groups of the profile, creating leaf objects for each in the data structure as they are found. After the data structure reaches a set capacity, it's compressed and sent to the server. The client then removes the obsolete data and continues crawling profiles not yet completed. Two questions arise from this approach.

First, how do we make sure we aren't wasting memory and a small amount of bandwidth by crawling profiles the server already has on record? The simple answer: by wasting an even smaller amount of bandwidth between the client and server to ask if the server needs the profile. When a profile is queued for crawling on the dataset, the client will send a message to the server containing the profile ID. The server will then respond with a message stating whether to continue crawling or remove the object from the queue. Herein lies the problem outlined before by our memory-bandwidth trade-off. On one hand, we could save bandwidth between the client and server by just crawling all objects put on the stack at the expense of the memory the objects would consume. On the other, we could save that memory at the expense of bandwidth to ask the server if all the profiles we queue have already been crawled. The problem is made even more complex by the thought that any profiles we remove from the dataset after sending to the server or after discovering it has already been crawled will not be known to the client in the future. The possibility of the profile resurfacing is almost definate (which is the purpose of the project to find out!). So, question two arises.

Two, how do we stop the client from crawling profiles more than once? As the answer to question one pointed out, we have the opportunity to just ask the server if we have crawled it before or not. But, this will inevitably turn out to be a gigantic waste of bandwidth because we will have to ask the server the same question more than once to receive the same answer. The alternative is to implement a low-impact cache, capable of quickly and compactly storing/recovering which profiles the client has been instructed are crawled. And, you guessed it, the trade-off is presented. The cache increases memory usage to relieve bandwidth usage.

This is a small glimpse into the current layout of the program. It most likely does not do justice to the project complexity, but should work to give you an idea of how the program will work and why it's taking me so long to complete it.

SixDegreeSteam Mailbag

2009-10-04T17:23:00.005-05:00

Any updates on SixDegreeSteam? You said you would keep us posted, but it's been a while since I heard anything.

- Anonymous

Work and college stuff has been keeping me pretty preoccupied, so I havent had time to do a whole lot with SDS lately. Im currently working on a low-bandwidth method of transferring data between the clients (slave threaders) and server. I also hope to post another article about the technical goodies behind the project, like what exactly a slave threader is. In the meantime, Ive been considering multiple web hosting opportunities to run the SDS server on. I think I will also use the space to start work on the full-fledged BlakeNet (under a different name, as I said before). I still have no word from Valve about using Steamworks or the acceptability of the bandwidth costs on their end. Ill have to shoot some more emails later.

Could I convince you to make SDS open-source?

- Anonymous

I have never been one to hold source code for personal projects from anyone interested in reading through it. However, I will have to keep it under wraps until development has ceased and I feel confident with its efficiency. Shortly after the project goes main-stream, the source code will be readily available for all to enjoy.

This article is a reply to an email. If you would like to ask a question or suggest a new article, email me. blake.oxx@gmail.com (remove the dot)

SixDegreeSteam Official Group Open

2009-09-12T22:09:00.002-05:00

The official SixDegreeSteam group (SixDegree$team) is now open for all interested Steam users to join. While project news and developmental goodies will be pushed here first, the SDS Steam group will be updated with important project updates only. Joining is a good way to get involved and show your support if you use Steam. Being a member of it will also grant you bragging rights when/if the project becomes mainstream, because members of the group will be the first to be crawled when we launch!

SixDegreeSteam: Challenges

2009-09-10T15:03:00.009-05:00

There are a few caveats to consider when developing this project. I have laid the information out in a (hopefully) easy-to-follow situation-caveat format. An important thing to remember is that not all caveats have an inherent solution, and the best solution is not always the most favorable.

Situation: First and foremost is the way Steam Community presents the information we are hoping to use. Since all information will be acquired by following "tips" from one page to a handful of others, and continuing on this pattern until all pages are analyzed, we call the process crawling. It works similar to a crawler for a search engine; Where a search engine crawler follows links on one page to others, our crawler will read IDs from one page and create new links pointing to sequential pages. Crawling friend information is easy, as the entire list of friends for a profile is in an HTML file. All we have to do is download the file from the site, extract all friend IDs, and queue them for crawling. Similarly, there is an XML file for each user containing profile information, including the groups the user is a member of. As far as crawling groups goes, a set of XML files for each group exists that lists all member IDs (one XML file per 1000 members). Note that the XML file we use to get group memberships of a user is not ideal, since it contains a lot more information than we need. This means that the speed of download and bandwidth usage (on both the crawling server and the Steam Community network) are impacted unnecessarily. A solution to this may be to use the Steamworks API available from Valve to developers, which features the ability to directly access community information without the need to crawl community pages, thus reducing bandwidth usage and overhead. However, Valve has declined granting me usage rights to Steamworks.
Caveat: Steam users have the ability to set privacy options that affect who is allowed to view their profiles. The current privacy options are private, public, and friends-only. Our crawler will only be able to use public profiles. Something to note, though, is that groups are always public, therefore all members are visible. This means that we can at least gather group membership information for non-public profiles. Likewise, if the private user is a friend of a public user, we will still be able to link the two users in our dataset since friendship on Steam is mutually inclusive (user A cannot have user B on their friend list without user B also having user A on theirs).

Situation: As was previously stated, the project is very web-intensive, utilizing a crawler technique to gather our data. This is a massive undertaking, requiring a huge amount of page/file downloads and subsequent parsing. While it is impossible to give an exact number to represent the size of our resulting dataset, the number of files we will need to download can be represented by a simple formula. We need at least one page per user (the profile information XML file, which also contains group membership information). This page is used to tell if the profile is public by checking to see if we get an access error, data is missing, or the privacy setting is explicitly shown. If its private, it would be wasteful to download the next file, as we dont have permission to view it and will just get a similar error to the first file. If its publicly viewable, we download a second file (friend list HTML) to complete our data collection for that user. Groups are a lot simpler as they are always public and will always have at least one member, therefore at least one file (the member list XML). Our formula to represent how many files we will need to download is A + B + (C * ceil(D/1,000)), where A is the number of Steam users, B is the number of users with public profiles, C is the number of groups, and D is the average number of members per group. As of this writing, there are over 53,300,212 Steam users and 1,101,806 groups. Guessing that at least three quarters of all users have public profiles and every group has 2,000 members (in reality, some users are only in one group where others are in one hundred, and some groups have one member where others have one million. 2,000 members per group is the best on-the-spot average estimation I can come up with for now), our file count is 53,300,212 + (53,300,212*(3/4)) + (1,101,806 * ceil(2,000/1,000)) = 53,300,212 + 39,975,159 + 2,203,612 = 95,478,983.
Caveat: To put that into data consumption terms, a member list for a group that has 1,000 members is about 42 KB, a friends list containing 50 friends is about 40 KB, and a complete profile page (used to get group memberships) displaying 10 group memberships is about 14 KB. Its very rare that a single file will be greater than 42 KB. Also, the user profile XML is what we use to tell if a profile is private or not, so its important to point out that the file returned for a private profile is about 366 B. So, modifying our previous formula to calculate bandwidth usage, we get (53,300,212 * 0.36) + (39,975,159 * (40 + (14-0.36))) + (2,203,612 * 42) = 2,256,007,309 KB = 2.10 TB. That is a serious amount of data to be transferring! We wont keep all of it, but as pointed out in Caveat 1, this is how we have to collect the data for lack of an alternative.

Situation: Caveat 2 points out that there is a lot of data transfer involved. So, bandwidth usage is definately an issue. Naturally, processing time for the data is also a factor. We cant provide accurate estimates for how long the data parsing for each of the three files would take because some computers are faster than others, processing time changes as memory becomes/is no longer available, etcetera. In either case, the obvious solution to speeding up both crawling and processing, as well as spreading bandwidth usage, is to use more than one computer. To speed processing up even further, we can create more than one instance of the program and have each instance work on a different page concurrently. This means we can process ten or more pages in the time it would take a single instance to process one. I will discuss both of these in-depth in a later article.
Caveat: Creating a control structure that can manage multiple instances (called threads) is one thing. Creating a server program that can remotely manage multiple programs running multiple instances is a completely different thing. The architecture of the server and its complimenting "slave threaders" will have to be such that each slave threader is capable of working, to an extent, on its own. The only time the slave threader should call home to the server is when the global scope of the dataset needs to be analyzed or all threads are idle (more work is needed). We adopt this minimalistic communication policy to make sure the bandwidth usage of the data between the two components does not become a factor in itself. This is a very intrinsic topic that requires extra thought in its own. As such, a future article will be devoted to this topic alone.

These are only a handful of issues we need to consider when tackling this project. They represent the major challenges I am facing right now in the design of the program. In an upcoming article, Caveat 3 will be examined more closely to show how I intend on structuring the server-slave threader relationship and my implementation of thread pooling.

Edited: This post was updated for correctness on January 14, 2010.

SixDegreeSteam: The Intro

2009-09-09T16:46:00.005-05:00

As my previous article hinted to, I am planning to write a program that expresses the link between every Steam user. As of this article's writing, there are exactly 53,300,212 accounts registered with Steam. This includes active, inactive, banned, duplicate, publisher/staff, and Internet Cafe ("multiseat") accounts. It is impossible to distinguish just how many of those accounts are actually worth the effort of this project without walking through each account profile (ironically, that's just what this project sets out to do). Each of those accounts is allowed to have a maximum of 255 friends, and an unknown (perhaps unlimited?) amount of group memberships. Now that we have some numbers down, let's talk about the actual program.

Introducing SixDegreeSteam, an attempt at mapping the Steam social network. The name is a reference to the Six Degrees of Separation sociological theory that the project is based on, which states that every person in the world is linked to every other person by no more than six people. My theory is that a similar phenomenon exists between all social mediums, specifically the Steam platform's underlying network Steam Community. However, there is a fundamental difference between Steam and other social networking mediums; Some people only use Steam to game, not make friends! This means that some accounts will have 100 friends and be members of 10 different groups, while others will have 0 friends and 0 groups. The network is even more obfuscated because some people may have friends and group memberships, but only with people they know well. This makes the web of connections incredibly shallow and in severe cases exclusive, with no connections to the mainstream web or other exclusive webs. In other words, "Everybody knows everybody" only works if "everybody" is not limited to a select few.

There are also a couple technical caveats we must address. They will be discussed in an upcoming article.

Six Degrees of Inactivity

2009-09-07T18:54:00.004-05:00

Where did you go??? It's been months since your last post! Are you working on anything interesting?

- Anonymous

First off, sorry for the inactivity, guys. Ive been insanely busy lately with a lot of stuff, mostly rectifying a couple situations in my personal life. Now, between college work and my new job, I dont have much time to myself. What little time I do have, I use to play games with friends or work on little side projects. Because of this, I wouldnt expect to hear too much from me for a while. Feel free to email me to say hi or catch up if youd like! I check my email daily when I get home from work.

Having said that, I do have a major project Im in the middle of conceptualizing. Circumstances providing, Ill make an in-depth post about it later. For now, the singular concept with have to suffice. Have you ever heard of the Small World Phenomenon, or more to the point, the Six Degrees of Separation model? They are very interesting ideas that attempt to link every person in the world. The idea was built on to create some popular games like Six Degrees of Kevin Bacon, Six Degrees of Wikipedia, and even a Facebook application. Surprisingly, there is no implication of this idea for the wildly popular Steam platform. Well, I think its about time someone puts an end to that, and who is more qualified to do so than me? :)

The goal of the project is to create a mapping showing how every Steam user is linked to everyone else. The project will take into account friends, as well as group memberships. This is a very exciting project for me and a major undertaking. More information to come when/if I get the time!

This article is a reply to an email. If you would like to ask a question or suggest a new article, email me. blake.oxx@gmail.com (remove the dot)

Multiplayer Ports

2009-05-30T11:18:00.004-05:00

Your multiplayer examples use port 8087. Is that required, or can I change that? Is there any importance to it?

- Anonymous

If you recall my multiplayer design series, the example code I published used port 8087. There is no real significance to this port. I just thought it had a nice ring to it. :)

In general, you can change the port to whatever you want. However, you should avoid ports that are used for other services. If the port is already in use, errors could occur. The ports used by system services and common programs should never be used. These are all ports in the range of 1-1023. To find a port that is unregistered or used by an exotic program, I use this Wikipedia article. As you can see, port 8087 is actually registered to some other programs, but they are not very common.

This article is a reply to an email. If you would like to ask a question or suggest a new article, email me. blake.oxx@gmail.com (remove the dot)

Carriage Returns

2009-04-11T10:44:00.003-05:00

Recently, I stumbled upon (literally) an article written by Chad W. L. Whitacre discussing a use for the infamously useless carriage return. Any programmer who has worked with direct input to/from file buffers has probably encountered a character that accompanies the newline character on some file systems. The usual sequence NTFS uses to terminate a line and move the internal pointer to the beginning of the next line is \n\r, where \n writes a null line terminator at the current position (effectively moving the pointer to the next line), and \r writes a carriage return.

If you stop to think about it, it makes sense. When you begin a new line, you want to make sure the internal pointer is writing to a fresh, blank line. So, first you move to the next line (via a newline character), then erase everything on the current line (via a carriage return). Beyond file input, Chad found that the carriage return could be used to erase data on the current line of a system buffer as well! In his article, he talks about using this technique in a TTY shell to make an updating progress bar. I thought I would take it a step further and see if it was implementable in a Java command line program. Sure enough, it works! The following is a basic program that uses a progress bar that updates as the program runs:

import java.io.*;

class progressbar
{
 public static void main(String[] args)
 {
  System.out.println("Running a really long loop...");
  
  // How many times the loop should run
  int runcount = 500;
  
  // Prepare the progress meter
  System.out.print("[0/" + runcount + "]");
  
  for (int i = 0; i < runcount; i++)
  {
   // Code that takes longer than a millisecond to execute goes here
   
   // Update the progress meter
   System.out.print("\r[" + (i+1) + "/" + runcount + "]");
  }
  
  System.out.println("\rdone!");
 }
}

Now no one has an excuse to not have progress displays in their programs! :)

TF2 Oopsies

2009-03-09T07:08:00.003-05:00

Lately, Ive been doing SourceMod plugin development for everything from infinite ammo/uber to new gameplay modes for use in BZG and other TF2 servers. This morning I was testing my newest gameplay plugin called Hot Potato. It was the first real vigorous test I had run with the plugin, and dramatically failed in flames of game crashes. I managed to capture a (laggy) piece of the carnage and post it on YouTube. Check it out. As the video description states, the plugin crashed at least four people, spawned 200+ entities within 10 seconds, and made 2244 function calls within 60 seconds.

I guess this serves as proof that pros are still prone to screw up. :p

The Brazen Guard

2009-02-16T06:20:00.002-06:00

The Brazen Guard (BZG) is this awesome online community I have been a member of since May 25, 2008. Currently, we operate two Team Fortress 2 servers and one Gmod server. We play strictly noncompetitive, although we do have a small group dedicated to clanwars and scrimmages.

Around January 24, 2009, I was given RCON administrative access and a place in the leadership of the clan. Since then, BZG has undergone some great changes thanks to a revitalization of the community. Something we do regularly is hold these things we call Event Nights. Every Friday and Saturday, we lock one of our TF2 servers and only allow BZG members in. Then, we play the game using our own little minigame ideas. Its absolutely insanely fun. Everyone has a great time and the brotherly bond we all share becomes stronger.

The whole purpose to this post is to inform everyone about BZGs own Youtube channel, The BZG Authority, where I will be posting video highlights of our scrimmages, Event Nights, and other madness. You can find more information about the servers and the clan at http://www.clantoolz.com/brazenguard. We are by no means an exclusive community, but if you would like to join, you have to play with us on our servers a couple times so we can be sure youre serious about becoming part of the family.

Anyone who has TF2 or Gmod, come on and play with us! Otherwise, enjoy the videos and be sure to rate/comment!

We're Back!

2009-01-07T20:10:00.003-06:00

BlakeNet is back online. This year marks the chance for a lot of new stuff on BlakeNet. The largest news is that I plan to start seeking sponsorship and analyzing development costs to upgrade the server, get some video equipment, and turn BlakeNet into the comedy video site it was invisioned to be. Dont expect to see these changes happen over night, but if all goes well, we might have an awesome new project on our hands! :)

BlakeNet Down

2008-12-23T00:27:00.002-06:00

BlakeNet will be down for a while. Our new computer just came in, which caused me to have to move equipment around and use the wireless device used to connect my server to make our second computer able to communicate wirelessly. Once I get back from Christmas vacation, I will start looking for a new wireless card so that I can get my equipment back online.

Merry Christmas. :]

Switching To Linux

2008-12-22T02:14:00.003-06:00

So, I got fed up with Vista recently. Well, if you know me, you know I have always hated Windows. Unfortunately, it often appears to be a necessity for certain situations. Gaming, MSN, school, hobbies, etc. However, as any seasoned Linux user would tell you, these lines of difference are becoming thinner and thinner all the time.

I have been using Ubuntu for BlakeNet ever since its conception, and I am no stranger to the environment. But, personally, Id just hate to convert my Vista laptop to Ubuntu and then not be able to do everything, and I mean EVERYTHING, that I used to be able to do. My plan of action? Well, I got annoyed at Vistas auto-backup mechanism, so I turned it off a while back and set up backups to my server. The 10 gig backup partition on my laptop has since been lonely without any data to store. I have decided to finally put it to good use. While I wont be uninstalling Vista due to the eminent possibility of headache, I will be installing Ubuntu on the backup partition, putting some apps on it, and using it as often as I can.

My setup will consist of Ubuntu Intrepid Ibex with Compiz (I looooove the shiny 3D cube desktop thingy!) and some apps to take the place of my Vista ones. Im not going to list off Vista programs and their Linux counter-parts. There are enough of those sorts of lists on the Internet, including the official one. My biggest questions were "will my games run?" and "will all of my favorite MSN features be available in the replacement messenger?" The first answer is, of course, yes with the help of the Windows emulation layer called Wine. But, I was worried about performance hits. From what Ive heard, the experience can differ, but most people say the performance is surprisingly similar (if not better!). I will have to update when the transfer is complete. The second question is due to my previous experience with Pidgin, the Linux Swiss-Army messenger that allows you to communicate on multiple protocols including MSN, ICQ, Yahoo, AIM, and more. Once, I had logged into MSN using it and found that it was an incredibly dry messenger with most of the features that made me fall in love with MSN gone. After doing some Googling, though, I found out that there are plugins that add those features back, including Messenger Plus! features! Needless to say, I was sold. I will be setting up for the next couple days, and will post my experience after its all done.

Update: Random Staffer

2008-11-28T04:06:00.003-06:00

Hey, peeps! I hope everyone had a good Gobble-Gobble-fest yesterday! Anyways, on to business.

Random Staffer is coming along nicely. However, I ran into something I really didnt like yesterday. In order to handle events on a form element (say, a button or menu item click), you need to use an ActionListener. There are two ways to do this effectively. The first, and seemingly most popular, method is to declare your class as extending the ActionListener class. This means you can point to this class as your listener, then add the ActionPerformed method to your class definition. This presents an issue of redundancy. For example, if you have some buttons and your class set up in the above fashion, you would need to add the action to take when the buttons are clicked separate from the button instantiations and set-up. This creates a readability nightmare, where if you want to know what happens when you click Button1, you would have to track it down in the ActionPerformed method further in the code.

The second method is to take advantage of Java's ability to change non-static method definitions at run-time. This means you can create a new instance of an ActonListener, and set the ActionPerformed method for that specific instance. This is great because you have resolved the redundancy and readability issues by including everything for the respective button in as little as two lines. However, this comes with the price of a new class file being generated for each time you do this, quickly changing a two-file project into a 20-file project. This happens because Java needs to create a completely new definition of the ActionListener class since the original has been changed.

Apparently, the desired effect is to have low-to-no redundancy, while eliminating the need for multiple class files. My proposal was to take advantage of Java's Class and Method classes. These provide your program with the ability to take in references to methods in other classes and call them dynamically. The problem I didnt forsee was that this only works on public and protected methods, which means every method that needs to be called when a button is pressed now has the potential to be called from the dangerous outside world. So, now its back to the drawing board.

BlakeNet Status Update

2008-10-17T02:54:00.002-05:00

The BlakeNet status to the right has been updated to fix the loadtime and caching issues. JavaScript is required to see the active status.

Random Staffer Reopened

2008-10-16T01:26:00.003-05:00

The old bane of my existence has been sitting on my todo list for a while now. The other day, I felt the urge to do something productive. Whats more productive than sleeping all day, you ask? Working on Random Staffer. As anyone who is a follower of my work knows, this program was a thorn in my side from day 1. It was originally built in Visual Basic .NET and the printing aspect was handled by Java. I decided this time around, I would rewrite the whole thing from the ground up completely in Java. Expect milestones to be posted as I go, along with maybe some source code and working pre-releases/finals. No ETC as of yet.

Battle Poverty One Webpage At A Time

2008-10-15T23:01:00.003-05:00

As a part of Blog Action Day 08, Im going to discuss how you can fight poverty. Since this blog more or less targets web designers, I figure Ill list some sites that offer buttons, badges, banners, and the like that when clicked or bought send money to charitable organizations.

MakeHope Button Store - The official button of Blog Action Day 08. Buy one for your site, and youll be aiding the movement.
Make Poverty History - A collection of free banners for your site.
Poverty USA - In case your visitors live under a rock, add these banners to spread the word.
End Child Poverty - In the words of the now defunct BannersAgainstPoverty, "Be the change you wish to see."

These are just four of the most popular sites. There are countless others to choose from.

Web designers, today we are called upon to do our part in the eradication of this world-wide pandemic. With the economies toppling, prices skyrocketing, and unemployment rates at all-time highs, it is up to each and every one of us to do the right thing. We owe it to not only those afflicted by poverty, but also ourselves, as human beings, to do all that we can to improve the quality of life for everyone -- A world for the people, by the people, for everyone's benefit.

BlakeNet Overhaul

2008-09-17T00:40:00.003-05:00

The classic green look of BlakeNet has started to bore me, and Im sure anyone else who visits the site. I decided that anyone who visits the site, especially potential employers (whom the site was originally intended to be viewed by), would find the bland, stinging default green a bit annoying and nubbish rubbish. So, I sat down and planned out a much more graphic-intensive, "cooler", engaging skin that shows off my talents a lot more effectively while still keeping to the techno theme of the site. And so, "Grunge Gears" was born. Today marks the full load-out of the new skin, along with a new skin selector right below the menu. (Yes, I plan to make more skins, and you can still use the old one if you need a low-bandwidth version)

I hope to hear some feedback about Grunge Gears. I have plans to change the menu from textual to images, create a new logo that follows suit with the skin, create some sort of filler image for the sidebar gap, and tinker with fonts to see which ones offer the best immersion. Besides that, drop me a line in the usual fashion if there is something youd like to see.

On a side note, sorry I havent been posting lately. Ive been really busy with job hunting, college stuff, and a cold/conjunctivitis/major ear infection triple-whammy. Just to reassure those of you who were hoping to see more articles in the Multiplayer series, I DO plan on writing at least one more, titled "Multiplayer Data Consumption". If I can get my life straightened out, Ill push it out ASAP.

Multiplayer Game Basics

2008-07-24T18:48:00.003-05:00

Multiplayer games come in all kinds of formats, genres, and languages. In this article, I will discuss the bare-bone concepts of a multiplayer game and even give step-by-step intructions for making a multiplayer backend (server) with Java. While I will only be discussing Java specifically, all the concepts apply to all programming languages. The general concepts require no real prior programming knowledge, though the Java tutorial accompanying this article requires a basic understanding of how Java works.

Multiplayer games can be put into one of two basic structures: Client-to-Client or Client-to-Server. In Client-to-Client games, the client communicates directly with another client, while in Client-to-Server, clients communicate with a server without ever having to talk directly to the other clients. This article will discuss Client-to-Server communications as this is usually the more preferred type for multiple reasons, which are outlined in the following table.

Advantages of
Client-to-Client vs. Client-to-Server

Client-to-Client

Low to no cost of operation to publisher
No reliance on publisher
Individual modifications

Client-to-Server

Security of clients
Slower clients dont slow down others
Centralized statistics
More quickly updated
Clients experience the game the way it was meant to be

All of these advantages and their inherited disadvantages can be traced to the simple fact that Client-to-Client games are individually owned and maintained by those who play them, where the Client-to-Server games are based around a "centralized" system of computers that are all maintained by the game publisher or others who are given permission to do so by the publisher. Before we begin the in-depth portion of the article, I want to clarify what a server is. A server in multiplayer games refers to a variant of the game or a related program that allows clients to communicate with it, and subsequently relay those communications to other clients who are connected to the same server. This web of client connections is called a network. Ironically, even in Client-to-Client structures a server exists. The server in this structure is more commonly referred to as the host, since their computer is not dedicated to serving the network.

Connections between computers over networks (i.e. the Internet or LAN) is usually accomplished with a system of sockets. A socket is a pipeline linking the server and client through a computer port. Each client shares a unique socket with the server, so no other clients can read data meant for other clients. Data traveling between computers is referred to as a packet. Most programming languages support the use of sockets and packets. The servers job is to listen to a socket, take in packets, do what needs to be done with it, and possibly reply to the client with another packet, or even broadcast packets to other clients currently connected.

Now that we understand the technology involved, its time to get down and dirty with Java. We will be creating a simple Java application that listens for incoming connections, designates the connection to a socket, and replies to each packet sent by the client after that. Programming in Java requires a text editor (I use good ole Notepad), the Java Runtime Environment (JRE), and Java Development Kit (JDK). Both the JRE and JDK can be downloaded from Sun's website. Java source code is saved with the extension .java. After compiling, the resulting programs are saved with the extension .class.

I have already programmed and tested the Java applications. You can download MplayServer.java and MplayClient.java from SNR, along with a simple client named MyTalker.java. To compile and run the server, navigate to the directory you saved the source files to with command prompt, then use the following commands:

javac MplayServer.java
java MplayServer 8087

To compile and run the simple client, use the same commands as above, except replace MplayServer with MyTalker.

Edit: The full step-by-step tutorial is done. You can find it at http://www.subnetroot.com/source/j/Basic/tutorial.txt.

The Importance of Pictures

2008-07-16T08:32:00.003-05:00

In presentations, pictures and visual aids are vital to keep the audiences attention and more effectively convey your points. In blogging, visual aids can be just as important. While the focus of blogging is on delivering raw information to the viewers, it can sometimes seem overwhelming to have an ocean of text with no means of visually presenting your information. I am admittedly guilty of this. I dont use images very often in any of my online pubications. In the future, I will try to include pictures in my blog posts so that visitors and subscribers dont get detered by all the reading.

On an unrelated subject, my last post on AI Design was the first in a series of entries about game design. The next one will be about multiplayer structures and will include an example written in Java showcasing how to use sockets to communicate between two or more computers. If you would like to suggest a topic for this series, or just something else youd like to read about, use the Contact Us form on BlakeNet or email me. blake.oxx@gmail.com (remove the dot)

AI Design

2008-07-10T05:28:00.010-05:00

I decided that Artificial Intelligence in the gaming world is something that is often glossed over when talking about game design in the sense that there are tons of pages detailing various dynamics of games a designer should consider, and even a couple discussing what exactly AI is. However, I am disappointed that there is a lack of publications describing the aspects of AI a game designer should know about when they decide to add some bots to their programs. While this post focuses on the impications, aspects, points of interest, etcetera specifically in AI for games, the concept can also be applied to Artificial Intelligence as a whole.

Lets start by defining what constitutes as Artificial Intelligence. AI is the ability of an object to make decisions that ultimately complete a set goal. This encompasses many things in a game, from bad guys shooting at you, to rival racers trying to beat you in a race. I like to split this definition into two types: Dumb AI and Smart AI. It might seem a little ironic that we call any amount of intelligence "dumb" or "smart", but it actually makes sense to classify AI as such. Imagine the classic game of Pong, where you play one paddle, and the bot (run by AI) plays the other. Each player tries to bounce the ball away from their side of the screen and past the other player. We classify this bot as having Dumb AI since it has a simple job of checking where the ball has moved to, and moving to match its position on the appropriate axis. There are no extra factors involved -- The bot doesnt check for where the player is, what kind of angle the ball should be bounced off at, or some other possibly critical decision-making datum. On the opposite side of the spectrum, imagine a game like Need For Speed, where the player is chased by police cars controlled by AI. This sort of AI is much more exciting in that the AI has to not only follow the player, but anticipate his next turn, dodge obstacles, and work together with other police cars to stop the player. This is a prime example of Smart AI. Dumb AI, while being effective in some situations, makes for very stale gameplay, or sometimes even frustration due to the bot being unbeatable.

Smart AI is what everyone wants to see more of. However, it comes with its own cons as well. A bot that is hardset on fulfilling a specific goal without factoring anything else in can lead to an unbeatable bot, which makes this type of AI just as terrible as the previous one. Since your average developer doesnt have a supercomputer residing in their moms basement, they arent capable of creating life-like intelligence that factors in human miscalculations and other realistic errors. This brings us to the next talking point; "Flawed" AI. To make sure a bots AI isnt infalible, many developers purposefully flaw their AI by adding a random chance that the AI will do the opposite of its goal, freeze up for a given amount of time, or something else that gives the player a chance to actually beat the bot. This sort of approach is completely in the wrong direction. The developer should not dumb down the AI for playability's sake. Now we get to the real meat behind this article. Lets examine two of the alternatives to purposefully dumbing down Smart AI, and figure out what Flawed AI really is.

First of all, consider what makes actual intelligence so manageable. Nature didnt create random counters that decide when an animal should eat and when it shouldnt. So what is stopping it from eating all of the food it can? The answer is a technique called prioritization. By prioritizing objectives, the animal can do what is most important at the time first, then doing the next most important thing. We can incorporate this into our AI in many different ways. Given that our game is a real-time strategy game like Age of Empires or Empire Earth, we can see how our gatherer-builders no longer gather until their "source of resources" is empty. Instead, they gather only when they need the resources and use the rest of the time to build vital structures.

Our second alternative is a business school classic called risk-benefit analysis. In this technique, the risk of a situation is compared to the possible benefits or perceived value. Risk-benefit analysis can be used in the previous example with similar effects. The further away a gold mine is from a guard tower, the more likely it is that the gatherer will be killed. In short, the further from defense, the higher the risk, and the less likely the gatherer will venture out to get that gold. To give another example, recall the Need For Speed police AI. The more police cars the player has damaged or totaled, the further back the other police cars will follow, and the less likely they will be to try to fish-tail the players car.

Flawed AI is an intelligent, objective-based approach to making AI more realistic and the gaming experience more enjoyable. Weve looked at just two alternatives to throwing our perfectly good Smart AI out the window. There are many more ways of achieving the same effect, but for now, Ill leave it at just these two. I might post about some more in a future article. Until then, enjoy building your worlds. If you have any games youre proud of, dont hesitate to post a link in the comments (Yes, comments are back up ;p ) or send them my way. blake.oxx@gmail.com (remove the dot)

HopeLine Promotion

2008-07-02T08:53:00.003-05:00

Im subscribed to The PostSecret Project. PostSecret is like a catalog of human emotion. Every day, random people send in anonymous post cards with their most closely guarded secrets on one side of them, and every Sunday, a couple are selected to be posted on the PostSecret site. It is a truely great concept and I believe its one of the best credits to humanity. A couple of my friends and I have sent in some secrets of our own. Its a great feeling to know youre being listened to without judgement.

Recently, PostSecret asked those of us with the ability to promote HopeLine to do so if possible. Im doing my part. All of my websites now feature the above video on the front page. Its my way of saying thanks. Thank you HopeLine, thank you Post Secret, thank you for all that you do.

Old Sites Going Up

2008-06-30T02:36:00.003-05:00

To make the Projects page of BlakeNet more useful, I decided to set up some of my old sites from the source code archives available from the Sites page to make fully functional recreations of the good ole sites featured on the page. I didnt think the screenshots and descriptions were cutting it. :)

Just a little tip to anyone bored enough to log in to the admin sections of the old sites and try to screw around with the databases or files, give up now. All of the database and file alteration functionality has been commented out. However, dont let that stop you from checking out the admin pages. They can all be accessed with the username 'admin' and password 'scinfaxi'.

On-Demand Graphing

2008-06-24T22:58:00.003-05:00

Something Ive been wanting to do ever since I learned how to use the imaging features of PHP is create a script that can take in some data and generate a graph on-the-fly. Useful? Definately. Invisionary? Probably not. In fact, Ive seen tons of PHP and CGI scripts that are used to generate graphs for things such as poll results. However, Ive yet to find a script that is open for public use or indy developer interfacing. Plus, most of them only do lame, boring graphs like bar graphs and pie charts. I wanted to introduce a script that can do them all, from bar graphs, to spider charts. In fact, the spider chart is the main focus of the project, and is the closest to being completed.

This script will appear in LeetBoard a lot, as well as an open-source model available on BlakeNet Source. No ETA yet, since Im still gathering up the types of charts I want to include in it.

BlakeNet Goes Live

2008-06-18T14:59:00.003-05:00

In another step towards making it a reality, BlakeNet is finally open to visitors.

The first complaint came in the other day about the performance of the Contact Us form, citing that it took a full minute or two to send messages from it. Upon investigation, I found that it was Sendmail's fault. My first instinct was to find an alternative to using Sendmail. I had the idea to do what I traditionally do with contact forms; Make them go to a SQL database, not mail them directly. Then I would set up a crontab that would mail me a digest of the SQL data every day or so. This would eliminate the wait to submit the forms, and would also make it easier for me to check and harder for me to avoid keeping up with (Yes, Im a very lazy person, if you havent noticed. As the quote goes, "all good programmers are lazy").

Today, I decided to take a whack at fixing Sendmail instead of setting up an alternative. My /etc/log/mail.log file had some interesting information. "My unqualified host name unknown" and "Unable to qualify my own domain name", to be exact. So, I guessed this was something to be fixed in my /etc/hosts file. I pulled it up and, after a couple tries, finally got everything running smoothly with no more complaining Sendmail. The line I edited now looks like this:

127.0.0.1 localhost localhost.localdomain blakeserver

The first bit is the loopback IP address, followed by the localhost host name, then the localhost.localdomain alias, and finally my servers hostname (If you dont know it, you can find it with the hostname command). Everything should work on your own machine, except the hostname. Hopefully, this can help someone out there with similar problems.

Starting Fresh: Day 4

2008-06-04T01:00:00.003-05:00

I reformatted and reinstalled Windows this morning. It surprisingly only took an hour or so. After that, I spent the rest of the day installing missing drivers, customizing settings, and installing and configuring security programs. I have to say it wasnt as bad as I expected it to be. I was expecting to have to spend an entire week getting everything back to my liking, when it actually only took somewhere around half the day. I give credit to nLite for allowing me to set up the installation ahead of time. Tomorrow, I will start reinstalling games and other programs I like to use.

This post concludes the Starting Fresh project. Good luck to those of you thinking of doing the same!

Starting Fresh: Day 3

2008-06-03T02:20:00.002-05:00

I took the last couple days to take a virtual inventory. I wrote down the programs I wanted to reinstall after the reformat along with any registration keys if they needed them. I also wrote down locations of files I wanted to put back on the clean machine from my backup, like pictures and music. This list makes it a whole lot easier to put everything back how you want it after reformatting. Remember that everything goes bye-bye when you reformat your hard drive.

If I can finally muster the nerve to go ahead and reformat, Ill start it tonight and reinstall Windows in the morning.

Starting Fresh: Day 2

2008-05-30T14:01:00.003-05:00

After two days of wrestling with Windows installation errors and unbootable CDs, I finally got the formula down for creating a Windows XP Home Edition bootable setup disk. Like the information I found online suggested, all you need to do it is an I386 directory, which is usually always found on Dell computers. Note that the I386 folder MUST BE THE ONE AT C:\I386. There is another similar folder somewhere in the WINDOWS folder, but that one doesnt have the same stuff the one in C: has. So, here are the steps I took to make the CD.

At this point, you might want to stop here and back up all of your files, including the OS files, if you havent already. If you dont back up now and decide to back up later on, youll probably be including all the extra programs and data were about to add. If you dont plan on backing up or want to just wait it out, read on.

Print out a report from Device Manager found in the Control Panel. This will help you identify missing drivers after you reinstall. This isnt required, but if youve got some extra hardware installed like graphics or sound cards, it can help out.
If you dont know your CD key, download Magical Jelly Bean to find the one used when your current installation was first installed. Write it down or print it out. I also printed out the other registration keys the program found, just to make it easier on me when I reinstall everything.
Create a folder in C: named XPSETUP. Now, put a copy of C:\I386 in XPSETUP. Make sure you copy the entire folder, and not just the files inside the folder, so that you now have an I386 folder inside C:\XPSETUP. I386 holds all the installation files needed by the Windows installer and is usually pretty big, so expect to wait a couple minutes while it copies.
Open Notepad and type "Windows " (including the space afterwards!), then hit Enter to add a new line. Go up to File > Save As, and type "C:\XPSETUP\WIN51" (including the quotes!) in the file name box. Click save and close Notepad. Navigate back to C:\XPSETUP and make a copy of the new WIN51 file called WIN51IC.
Now the tricky part begins! If you know what Service Pack came pre-installed with your computer, youll be fine. Otherwise, youll probably have to shoot-and-miss to get the disk to work correctly. If your computer didnt come with any SPs pre-installed, skip to the next step. If SP1 came with it, make a copy of WIN51IC named WIN51IC.sp1. If SP2 came with it, make two copies of WIN51IC named WIN51IC.sp1 and WIN51IC.sp2.
At this point, other web sites would suggest you slipstream the newer service packs. I decided not to after getting a headache while trying to get the disk to work with the slipstream. You can look elsewhere if you want help slipstreaming, but dont ask me for it. I couldnt figure out any easy and fool-proof way of doing it.
Download nLite and use it to customize the installation. This is optional, but if you dont customize, at least use it to create the ISO.
Use your favorite CD burning program to open the ISO created in the previous step and burn its contents to a fresh CD. Congrats! If all went well, you are now the proud owner of a Windows XP Home Edition bootable setup and installation CD.

The next task is where the real fun begins. Reformatting and reinstalling everything. Im not looking forward to it, because Im so lazy. But, Ive come this far, so might as well continue!

Starting Fresh: Day 1

2008-05-28T12:44:00.003-05:00

The first official step towards this, in my opinion, huge undertaking is backing up the entire hard drive onto my brand-spanking-new 1 terabyte Western Digital My Book. The backup itself only took around an hour and a half to copy all 83 gigs of data. I simply booted up to my Ubuntu LiveCD, mounted the backup drive, and used the cp bash command to copy everything over. Afterwards, I ran the diff command to compare the backup and original just to make sure everything was copied correctly. There was only one mistake I had to correct manually, which was a file missing from the backup that was originally named with a tilde (~). Apparently, that character messes up copying or something.

The reason I copied everything from Ubuntu and didnt just use the Windows Backup utility is because I wanted EVERYTHING, not just my data, so that if I mess anything up, I can just clear off the hard drive, copy everything back onto it, and pretend the whole thing never happened. This admittedly costs a lot more space, but if youre like me, you dont want to have to reinstall and reconfigure everything if you screw up, with the only consolation of your backup being you didnt lose any of your personal documents.

The next step is to create a Windows XP Home Edition installation CD. I found some scattered guides on the internet that touch on how to do it, but Im thinking its going to be shoot-and-miss for a while. Ill detail the process I took on my next post.

Reformatting/Reinstalling My Desktop

2008-05-27T02:49:00.004-05:00

Ive been wanting to do it for a while, but today I finally bought a 1 terabyte external hard drive. My first act with it is to back up everything from my desktop computer. Its a fairly old machine, running Windows XP Home Edition. Over its lifetime, Ive seen its speed decline drastically. Now, those of you who keep up with my work or know me personally know I am not a noob with computers. Having said that, Ive tried all the tricks up both my sleeves to get it running faster than the painful speed its currently at. Nothing has worked. So, Ive decided its about time to reformat, reinstall, and start fresh. Its a big step, but hey, Ive got a whole Summer of boredom I need to kill. I might as well do something productive.

The fun begins with my lack of Windows XP installation CDs and keys. Luckily, I know how to retrieve keys of already installed distros with the help of Magical Jelly Bean. The CDs, however, are semi-new territory to me. Ive been informed that theres a way to create setup cds from the otherwise meaningless i386 folder found on a majority of computers.

Ill be posting my progress as I go, and explaining what Ive done each step of the way, as Im sure there is some poor, lost soul out there in the same predicament Im in. All progressive posts for this project will be titled "Starting Fresh" for ease of access.

How To Change The Windows Explorer Start Location

2008-05-06T20:29:00.005-05:00

NOTE: For some odd reason, this does not work on some systems. Im trying to figure out why that is, but until then, this guide should only be brain food and shoulnt be expected to work. Sorry!

After Googling around and not coming up with any indications as to how to change the location Windows Explorer automatically opens when you right-click the Start button and choose Explore or Open, I decided to do some digging through the registry and found out how to change both of them.

If you are a hands-on person who isnt afraid to mess around with your registry and possibly screw up your computer, open up the registry editor (type regedit into the run box and start it), then navigate to the key HKEY_CLASSES_ROOT\Folder\shell. In this key are two subkeys named explore and open. Select the key you want to change and then select the command subkey. In this key, you will find a (Default) value. Edit it to whatever command-line command you want Windows to run when you click the option in the right-click menu.

While this allows you to use any program instead of just Explorer, its still recommended that you keep using it (at least for one of the options) for support reasons and old times sake. If you need a list of options supported by Explorer, you can find them here or in the incomplete Microsoft list. If you are like me, you just wanted to change the location it starts up in and have no interest in all the other options. In that case, you can just copy the following:

%SystemRoot%\Explorer.exe /separate,/e,C:\Program Files

Replace the bold location with the location youd like Explorer to start in. This registry hack is based on Windows Vista, but Im fairly sure itll work in XP as well. In case you dont like the change, the default value for Explore is %SystemRoot%\Explorer.exe /separate,/e,/idlist,%I,%L and Open is %SystemRoot%\Explorer.exe /separate,/idlist,%I,%L.

Imaginary Files

2008-03-30T00:57:00.002-05:00

Today, I was doing some work with GIMP on my Windows Vista laptop. When I went to save an image, I noticed a folder in the Program Files directory named HTTP-Tunnel. I guess this was a leftover folder from when I uninstalled the program of the same name earlier this month. So, seeing how the GIMP file browser isnt capable of deleting files and folders, I opened up Explorer and navigated in the folders direction.

To my surprise, the folder wasnt there! To confirm it wasnt an issue with GIMP, I started Notepad and tried to navigate to C:\Program Files\HTTP-Tunnel, only to have Notepad complain that the path didnt exist. Next, I opened a command prompt and ran the dir command on Program Files, which also claimed that the folder didnt exist. My next thought was that HTTP-Tunnel had been mistakenly classified as an imfamous super-hidden file, a junction, or something of the same behavior. To make sure, I ran dir again with the A flag ("dir /A" displays all the special file system objects that are usually skipped over in order to keep average users in the dark about special OS-related functions and the likes). This also didnt bring up any trace of the directory.

My final thought was that GIMP was using some sort of cached display to spare the program from having to repopulate the directory view so many times. This was easily proven wrong when I browsed to HTTP-Tunnel in the GIMP explorer and found it contained a text file. I searched my C: drive for the uniquely named log file, and came back with one result. Apparently, there was a folder in C:\Users\Blake\AppData\Local\VirtualStore\Program Files named HTTP-Tunnel, which contained the file in question. I deleted the folder, and lo and behold, the folder vanished from GIMP.

Im not sure yet why, but Vista seems to sometimes save copies of files, folders, or entire directory trees in C:\Users\%USERNAME%\AppData\Local\VirtualStore. Ill have to look further into it, but my guess is this is where files that are accessed a lot are temporarily stored to be found easier by programs, or where data is moved on special occassions when you delete things from the Recycle Bin.

The Hesitant Scrapper

2008-02-23T20:45:00.002-06:00

I have an old machine that is far too outdated to become an efficiet bulldog system, so I decided to rip the hard drive out, disassemble it, and tie some string around the three metal discs inside it to make some pimping bling. This machine is the same one that you might recall reading about having its networking card ripped out for my server project. Having served its purpose, its destined for the curb. But, I read something today that made me wonder if I made the wrong choice in badgering the hard drive. I completely forgot about the possibility of using it as a back-up system for one of my other computers. So, Ive decided to attempt to put it back together. It will surely lead to either frustration or, if I do make it suitable for plugging in again, decapitation. Still its a fun little task to keep boredom at bay. I also have an even older machine that is so old it boasts an LCD numeric counter, reset, turbo, lock, power button, and cigarette lighter on the front of it. Ill be pulling resources out of it next.

By the way, before you suggest it, yes, I already checked the first machine for salvagable memory. Nothing special in that department, though I was intrigued to find the mother board had three memory slots where my newer machine Im using as my server only has two.

ROOTrants

2008-01-26T15:46:00.000-06:00

Ive created a second blog dedicated to my rantings. If you want a good laugh, or a dose of reality how I see it, check out ROOTrants. Enjoy!

New BlakeNet Developments

2008-01-21T17:16:00.000-06:00

Today I finally got around to setting up a hostname for BlakeNet, so no more having to memorize the IP address or worry about it when it changes! The new address is http://blakenet.no-ip.org.

I also recently reconfigured LeetBoard and set a full distro up on BlakeNet for all intensive purposes. Some things broke during the transfer, but for the most part its a working project again. I would give an address for it, but then Im sure spam would start flying at it from every angle, and lead to a gunked up server. I will release an address once I have it running stable again.

Also, welcome to the team a new Netdev. Nathan (Squirrels) will be goofing around with the box just to try and learn something new, or fend off boredom. He may also be helping me further develop the current and future projects featured on BlakeNet.

Next Task For BlakeNet

2007-12-23T10:25:00.000-06:00

In case some of you havent been keeping up, BlakeNet is the webserver project Ive been using to test and build my various programs, especially LeetBoard. However, a new idea has been discussed recently. A while back, my good friend affectionately nick-named Squirrels and I had talked about how great it would be to start a website where we could just put stupid stuff to make other people laugh and generally have fun with it. We never got around to purchasing hosting and creating the site.

But, last night, he mentioned the idea again, and I suggested we use BlakeNet and purchase a domain name, which is a much cheaper solution compared to buying hosting. We arent sure when we will be doing it, but hopefully this time we will get around to setting things up. The name of the site is still undecided. As for what it will be about, well, Ill keep that a secret to build suspence.

Building A Multi-User Application

2007-12-15T10:23:00.000-06:00

Ive been working on a little project lately to test communication between one computer and another through the internet. This was a great opportunity to use BlakeNet. The plan was to host a daemon process that listens on a given port, accept incoming communication requests, and reply to messages sent to the server. The server program was programmed in Java, and the client program was originally programmed in Flash, but later I decided to create a second version in Java.

Working on this project brought up an interesting quirk about multi-user applications, though. It turns out, in the case of Java, at least, that ports can only have one persistent connection at a time. All other connection requests on that port are queued, and are only allowed to open once the current connection closes. There are a couple different ways of getting over this problem. Im still considering which one to implement.

The first is to have the server continue to listen on one port, and have the server constantly kill the current connection, accept the next one in the queue, communicate, kill it, and repeat. This means the clients, in order to have semi-persistent connections, would have to send in another connection request each time the current one is killed. The problems with this is it has the possibility of being very bandwidth- and cpu-intensive, may lead to "greedy clients" who skip ahead of others in the queue and cause them to get more attention than other clients, and might lead to a loss of clients since theres a limit of 50 (or less, depending on the operating system) connection requests per queue.

The second is to have the server listen on one port for connection requests, then assign each request to its own port. This is probably the best way to go, but it does have its pitfalls. The biggest one is that having so many ports open, and only having one program to maintain all of them, can be a disaster to processing speed. The other, less significant problem is that there are only so many ports you can use. Ports 0-1023 are registered for use with system protocols and require superuser privileges to use, and ports 1024-49151 are registered for use with commercial programs. This means, under the worse-case-scenario, only 16382 or less connections can be allowed (ports 49152–65535, minus one for the connection request port). That is way more than needed, probably. But, limits are limits.

The third is to just create a peer-to-peer style application. The server will only be responsible for registering clients, and telling other clients of where to find them. All actual communications will be done between the clients themselves. This can lead to the client computers lagging and also leaves a lot of things up to the client to configure, especially security-wise. This is also similar in structure to the first proposed solution. So, this choice is definately out.

I think the best way to go is the second choice.

BlakeNet Up Finally

2007-12-06T17:33:00.000-06:00

Im proud to say BlakeNet is now finally up! It is still under heavy construction. As of this posting, it only has a placeholder index page. Expect to see LeetBoard up on it, along with possibly a working website just for the fun of it!

BlakeNet will not be up 24/7. As such, Ive added a new item to the sidebar that lets you know if the server is online or offline. When it is up, you can access it by clicking the link in the sidebar, or going to this IP address. Right now, it is only accessible by IP. I might buy a domain name for it later, but it isnt really on my todo list.

Back From Georgia

2007-11-27T00:34:00.000-06:00

I came back home from Thanksgiving break in Georgia yesterday. Surprisingly, I really enjoyed the time with the family. I hope everyone had a nice Thanksgiving as well!

I brought two things home with me from Georgia. One is a little 6-week-old kitten we named Gracie (I might talk about her more later, or even post some pics), and the other is an old computer my aunt was planning on scrapping. It has some interesting family tree info on it she wanted me to pull off of it first, but after that, it is all mine. The reason I decided to take it home is because it has a PCI Ethernet adapter, which if you recall I have been looking for to connect my server to the network/internet. I will be doing some more work with the server and seeing if I can get the adapter working with it over the next couple days.

Me? A Teacher?

2007-11-15T23:27:00.000-06:00

Earlier in the year, I suggested the idea of teaching GML (Game Maker) to the Programming II class. The teacher, Mr. Allen, decided that the first semester, he would teach the class Java, and the second semester, I could teach the class GML. Today, he gave me a 17-day timeframe to come up with a basic lesson plan to use after Christmas break. Im so excited to finally be able to share my infinite wisdom of computers with a class. :p

If I come up with a lesson plan that I feel like sticking to, Ill be sure to put it up on BlakeNet, along with select projects created by the students of the class.

BlakeNet

2007-11-14T19:55:00.000-06:00

Saturday, November 3rd, I got bored and decided to completely clear off one of my old computers, install Linux, and get a server going. The next day, I installed and configured Apache, PHP4, and MySQL. What took me two weeks to post this? Well, Ive been busy working with the hardware of the machine.

The computer did not have an Ethernet adapter, so I got one from one of my dad's old work machines. The problem is the adapter is ISA, which Linux doesnt exactly like, and the ISA port seems to have issues with the adapter plugging into it fully. Either of these could be the cause of the problem. So, after two weeks of fighting it physically and digitally (with the immensely appreciated help of the LaunchPad help group), I have finally reached boiling point and trashed the card. Now, Im looking for a PCI Ethernet adapter.

In the meantime, I will be working offline with LeetBoard and a couple of my site concepts. When I find the adapter, you can check out the progress first-hand. The server, and inheritantly the website from which all of this material will be available, will be refered to as BlakeNet. Yes, its a pretty lame name. Ill come up with a better one when I have time to stop and think about it.

Playing The Oldies

2007-11-03T22:23:00.000-05:00

Well, I decided to head into my room today and untangle the mass of cords all huddled up in the corner. After sorting them all out, I got to touch my old SNES and NES from way back in the day. I dusted them off and hooked them up, and sat down, ready to play.

Unfortunately, I couldnt get the SNES to play on my TV correctly. Ill have to work on that more later. But, I DID get the NES fired up, and after a few savage beatings, I actually got to play some of my old favorites. Super Mario (one and three), Mission: Impossible, Star Tropics, and even Solstice! It really took me back to the good ole days. I plan to take a break from the Wii for a while to just take in all of the glory of the old, crappy games we used to play and love to remember.

If any of you have an old system that may still have a chance of functioning, I suggest you wrestle it out and play it for a while too. Halo 3 wont miss you that long. ;)

MyishiCorp.com Down

2007-10-21T13:02:00.000-05:00

Today, MyishiCorp.com reached the end of its lifespan. This includes the LeetBoard beta testing. Myishi vets can still use the old forum, although it may not be very active. We encourage all vets, newer members, and old allies to get MSN Messenger and keep in touch with us through it. If you have it, but dont have any of our addy's, e-mail me.

The shutdown of the LeetBoard beta will drastically affect the development of the software. I will be setting up a new site for testing soon enough, but for now development of the project is suspended.

It's been fun!

Recent LeetBoard Updates

2007-10-10T23:26:00.000-05:00

Lately, I have been working quite diligently on LeetBoard. As a result, four new features and fixes have been implemented today.

The first is the moderator macro feature. This allows administrators to set up shortcut operations, or macros, for board moderators. The macros can pin, lock, move, and reply to topics. Macros can be limited to a specific forum or forums, usergroup, or a combination of the two. In the future, the power of the mod macros will be expanded to allow actions to be taken on the posters themselves.

Second, we have polls ready for use in topics. Polls can be appended to regular topics where they are allowed. You can read more about this feature in the previous post title LeetBoard Tech: Polls.

Next up is the post title/description issue. It turns out there was an error with how the title and description of new topics were being processed, causing them to display strangely if certain reserved characters were used. This also lead to both fields being cleared automatically upon previewing the new topic before posting.

Finally, another fix for the smiley system. This issue involved the "Show All Smilies" link missing from the Editing Topic form.

I will continue to work on the LeetBoard software. Each day it gets closer to completion. A special thanks to the Myishi Corp community for their continued help in beta testing the software and feedback. For additional feature updates, check the todo list.

LeetBoard Tech: Polls

2007-10-10T23:17:00.000-05:00

Many message board softwares allow their users to create polls that allow other members of the board to read a question and decide amoung several choices provided. LeetBoard now supports this same feature. A user can provide a poll question, along with a minimum of 2 or maximum of 6 options, to be appended to the top of a topic. The options may contain BBC and smilies. After voting, the users can then view a pie chart rendered on-the-spot that shows the current poll standings including the total votes, and number and percentage of votes per choice.

MC Closing Again

2007-10-04T20:51:00.000-05:00

Well, it seems Myishi Corp is going down for the count yet again. This time it will probably stay down. Even though I sort of expected MC to run out of steam so early in its revival, it's still pretty sad to see poor ole Myishi going out for the, what I guess to be, third time in it's on-off life.

As there is no point in having a website for a corporation that no longer exists, MyishiCorp.com will be dissolved after the latest payment on the hosting and registration expires (which should be sometime near the end of October 2007). This means the forum will also be pulling a vanishing act, which will effectively end this round of LeetBoard development. I will be starting a new website somewhere down the line where I will continue development of the project. But, until then, the only place you will be able to see anything LeetBoard-related will be here. For anyone who is reading this and was a former member of MC, the old forum will still be up, although I will not be actively administrating it and very very few others will be using it. We prefer you get MSN/WLM and chat with us using it instead.

Retargetting Images

2007-08-27T20:12:00.000-05:00

I just had to blog about this. This technology is awesome. Basically, its like resizing an image, except instead of just subtracting rows and columns of pixels, the program seeks the best pixels to remove, that way the image doesnt appear lop-sided or otherwise deformed. It also works for enlarging an image. Check it out.

Image Resizing

LeetBoard Tech: Custom BBC

2007-08-24T20:35:00.000-05:00

LeetBoard aims to provide you with the ability to create your own BBC tags to use on your board. Unfortunately, this means the mechanism responsible for creating the buttons users click to add the BBC to their posts requires some advanced programming in order to allow you the freedom to create BBC tags without any restrictions on how much they can do.

BBC tags are traditionally just short-hand codes which are replaced by their HTML counterparts. But some more advanced tags may allow you to use syntax highlighting, or even generate dynamic content. No forum software before LeetBoard has given you this level of power when creating custom BBC tags.

LeetBoard Tech: Modular Scalability

2007-08-18T13:55:00.000-05:00

Something that is in high demand in the computer industry nowadays is modular software. You want to be able to quickly and easily add a new feature to your software without having to drudge through millions of lines of code. Many forum software packages give you the ablility to add your own codes to the mix. But, the problem is many of them get a little carried away with it all. Take MediaWiki or phpBB for example. Sure, they are completely modular, right down to the error message texts. But, they are just TOO modular. It's very easy to get confused or turned backwards in the millions of lines of code you have to go through just to find where to put a new feature.

LeetBoard is not modular to the extent of MediaWiki or phpBB, but is modular enough to allow you to easily find where a new feature needs to be inserted or create a new feature for your board without needing to step through twenty different pages of code. All major features of the board have their own independant file, which is layed out in a manner that is easy to follow. If you want to create an entirely new major feature, just make your own file for it and drop it in the custom module folder, and LeetBoard will automatically use it when it is requested by a user.

LeetBoard Tech: Logging

2007-08-18T13:45:00.000-05:00

All forum software big and small is prone to "trashings". Whether it's Invision or phpBB, you are almost guaranteed to have an incident where a moderator account is stolen or a moderator decided to go AWOL and delete a couple or all of your board's posts. No forum software has ever featured the one thing that could save you from such a disaster; that is, until LeetBoard!

LeetBoard boasts a logging feature that records all moderator and administrator actions and automatically generates an undo code, allowing you to restore a board to its former glory with one click. This feature is completely new and can ONLY be found in LeetBoard. Just another reason why you may want to switch your forum software to LeetBoard when it is publicly released.

LeetBoard Tech: Searching

2007-08-18T13:38:00.000-05:00

Creating a search engine is very simple. Creating a search engine that has all the features a user would ever want is another story entirely. At present, the LeetBoard search engine only allows a couple of basic searching techniques. It would take forever to make the search engine as advanced as Google, but we would like to continue working on it until we have a respectibly close one.

LeetBoard Tech: "Is Read?" Issue

2007-08-18T13:21:00.000-05:00

A feature that is familiar to everyone who uses any sort of forum software is the "Is Read?" or "Is New" feature. This feature is responsible for telling you what posts you have and havent read every time you log into or visit a board. Every forum software has a different way of accomplishing this. LeetBoard uses its own unique style, utilizing a database and cookies.

When you first log into LeetBoard, a blank cookie is set. When you read a topic, the ID of the topic and its newest reply is saved to the cookie. If a topic was made between your previous login time (your last login) and the time you just now logged in, and that topic does not appear in the cookie, the topic is marked as new. Similarly, if a new reply was made between your previous login and your recent login, and the parent topic's ID and the reply's ID dont appear together in the cookie, the topic is marked as new. Logically, this should be a perfect method for checking if a post is read or not, but for some reason it does not work correctly. I am not sure why this is, but I am currently working to figure out what the problem is and fix it.

Edit: The problem was due to a conflict with how posts were decided as being read or unread. Each member has a "last login" time saved with their account. This time is saved each time the user logs in, and its previous value is used to see if posts were made between the previous last login and the current time. The issue has been resolved by adding a query that updates the last login time to the current time when the user logs out. Issues may still occur if the user explicitly deletes the cookie which stores the forum session or lets the session expire without logging out.

LeetBoard Tech: Smiley Issue

2007-08-18T13:04:00.000-05:00

An issue has been found concerning the smilies. When rendering the smilies from textual to graphical, some smilies are not correctly rendered. For example, if we have two smilies available, :( and >:(, and we have a post containing the text ">:(", the sad smiley is rendered with a preceding greater-than sign (">").

This is called a collision; when two conditions are similar, and one is checked before the other, the other is not met or is overridden. It is due to the renderer checking the whole text and replacing all occurrences of the sad smiley then checking the whole text and replacing all occurrences of the angry smiley. In order to fix this issue, the renderer must check each character of the text one by one and compare it to all possible smiley conditions. This is rather tedious and will add anywhere from half a second to two seconds, depending on how many smilies your board has, to the renderer's runtime.

Edit: All current smiley-related issues have been fixed. Smiley functionality is now complete.

LeetBoard Status

2007-08-03T05:52:00.000-05:00

I didnt get a chance to post about my newest code back when I started work on it, but heres a little catch-up post. Ive created some forum software affectionately called LeetBoard. It is actually a further build on a bit of forum software I designed about a year about for a gaming clan called ThE NuBs, at which time the software was called ThE NuBs Chat.

Now, it is up and running for the Habbo Hotel in-game corporation Myishi Corp and is a prime feature of their site. You can find the Myishi Corp website, also done by me, at MyishiCorp.com. LeetBoard is currently in beta stages of development as a private project, and is just now on the verge of entering version 1.2. However, upon completion, it will be made public.

My Space on MySpace

2007-05-20T22:04:00.000-05:00

Well, Matt threatened to make a MySpace for me that would make fun of my mother, or something. So, I decided to make one just to keep him from doing one for me. You can find it at this page.

Dont bother adding me to your group or friend list or whatever, as I wont go on it much. Its really a terrible website. I dont understand what everyone else likes so much about it. Well, at any rate, enjoy!

Update: Warn Logger

2007-04-22T06:38:00.000-05:00

Ok, after roughly 12 hours of the release of Warn Logger v1.3, its now time to release the second edition to it. Introducing Warn Logger v1.3.1!

The only added feature is the long-awaited "New Topic" mode, in which logs are saved as individual topics instead of replies to an existing topic. Along with this mode comes some dynamic tags used to show helpful data to the administrators who review the logs. Check out the installation instructions for more info.

Update: Warn Logger

2007-04-21T18:25:00.000-05:00

Warn Logger v1.3 was released today. It fixes a few compatibility issues with FireFox and actually works a lot smoother with both FF and IE now. One of the major issues fixed was the bug with the onLoad event.

The Frame responsible for loading post data and external links during Warn Logger's execution is required to load a page and let Warn Logger know when the page is done loading and execution can continue. The easy way of doing this is to use the onLoad event. Unfortunately, the onLoad event has some serious issues, especially when trying to dynamically change the function it calls. So, I swapped out the onLoad event with a substitute function that seems to work just as well. Here's the function:

function waitForLoad(nextcall)
{
// Since there are issues with the onLoad event, this function recurses until the document has no updated content
if (currentdoc == ""){ currentdoc = window.frames['wlframe'].location.toString() }

if (currentdoc == window.frames['wlframe'].location.toString()){ setTimeout("waitForLoad('" + nextcall + "')", 500) }
else
{
try
{
if (olddoc == window.frames['wlframe'].document.body.innerHTML){ currentdoc = ""; olddoc = "" }
else{ olddoc = window.frames['wlframe'].document.body.innerHTML; setTimeout("waitForLoad('" + nextcall + "')", 100) }
}
catch(e){ setTimeout("waitForLoad('" + nextcall + "')", 100) }
if (currentdoc == "" && olddoc == ""){ eval(nextcall) }
}
}

I call this function and pass the function to run after the page is loaded to it. Then, I change the frame SRC (document.getElementsById('wlframe').src), and ta-da! A custom onLoad event! Feel free to use this code as you please.

A New Support Approach

2007-02-12T19:39:00.000-06:00

Howdy! Well, it's time to blow the dust off this blog with another new idea.

Ok, so, I've been getting a lot of support requests lately from people with problems with good ole Warn Logger. Now, I love helping people out and am in no way trying to tell people to back off with this concept, but the same problems keep reoccurring. So, heres what I plan to do.

I'm going to design a new chunk of code to allow the user to troubleshoot the code before they contact me. The troubleshooter will allow the user to fix the problem theirself or tell the person exactly what kinda information I need to get from them in order to fix the problem quickly and efficiently.

Here's a run-down of how I immagine it to work. The code starts by writing a try block on every line. If an error occurs, the try block will know of it and alert the user. The code will also be aware of the common errors encountered and common remedies the user should perform. The code will also check to make sure the code to test is installed properly. If all else fails, the user will be alerted with run-time reports from the code that need to be sent to me in order to fix the problems.

It's a bit low-tech, but I'm thinking up different ways of going about doing this without all the try blocks and such. If you see any codes pop up utilizing this concept, guess who the coder most likely stole it from! :p

Code Census

2006-10-16T00:34:00.000-05:00

Wow, its been a while since Ive updated the blog. Ive been busy with a bunch of things.

Recently I launched a code counter with all of my public InvisionFree codes (as of this posts writing, three) to get an idea of just how many people use my codes. The counter was left on for a full 24 hours, give or take a few minutes. Im pretty happy with the results of the census; approximately 150 individual active boards have my codes installed and working. I say approximately because the results included about 2 google caches and 4 duplicates, and also did not count any of my codes which are self-hosted by the board owner. The next time I activate the counter, I hope to see even more boards in the results. The counter ate my bandwidth a bit, so I cant release a link to the results at this time. Sorry.

Also, I just updated to Beta Blogger and added an AdSense bar.

Two Code Updates for Reply-Blocker and dSig

2006-05-20T16:13:00.000-05:00

Reply-Blocker has a few updates in store for it. Most of them are similar to the ones applied to dSig. They are listed below:

Make individualized (private per-user) cookies
Fix manager feature
Fix loadpoint
Confirm compatability with FF
MAYBE disallow Mod/admin blocking

dSig will also be receiving a small update to its cookie feature. The cookies use a character to distinguish different users and blocked names. The problem is if any user has that character in their name they cant be blocked and also cant use dSig! My solution is to change the character into something that IF boards dont allow to be in their user names. I havent figured out what yet, but when I do Ill let you know. This update will be included with the new version of Reply-Block as well.

Finished: dSig Update

2006-05-09T18:34:00.000-05:00

I released dSig v3 about a week ago. It was going well until I got reports of an incompatibility with Firefox. Today I pushed out dSig 3.0.1, which features support for Firefox and a new runtime expansion that allows dSig to run faster than before.

The main reason behind the runtime expansion was not a concern of speed, but of conflicts with other codes. dSig uses a User CP addition called dSig Manager. In the method before 3.0.1 the code was run through the window.onload event, which can only hold one value per loaded page. Meaning if there is another code using the onload event, only one of the codes will run. In the new method, I check to see if the page is already loaded. If it is, dSig is run immediately. If it isnt, dSig is added to the onload event. I would like to make the immediate running mandatory, but that would mean becomming unavailable to boards with previous versions installed.

Reopening dSig

2006-04-23T13:11:00.000-05:00

Well, Ive been working on the forum software for ThE NuBs. Its still not completely done, but its getting closer. I also finished Project Apex for now till the owner of it gets online again.

So, with those two monster projects done, Im ready to reopen an old code: dSig. I had said I would upgrade it back in the days when I wasnt so busy. I tried to open and edit the beta files of the work I had done towards the new version, but after reading through it a few times, I realized that the function and methods I had used in the old version wouldnt be able to work correctly with the ammount of data I would be passing it in the new version. As a result, Ive decided that it will be easier to completely recode the new version instead of just changing the structure of the function.

That means it will be a longer wait for the code, but will be definately worth it. As usual, Ill continue to post updates on the new versions progress as it unfolds.

Warn Logger Trouble

2006-04-05T18:06:00.000-05:00

At first, the release of Warn Logger seemed like a big success. But then reports came in of a problem that was causing a board error. After cranking out the next version with the fix, I found out that for some reason one of the features, the one that allows you to post reports as new topics, wasnt working. I have no idea why that is as the code that deals with that feature was never even touched.

With this version came two more problems! The code now had some compatibility issues with Firefox and was causing "Aborted Action" errors in Internet Explorer. After talking with a friend, I figured out that the IE error was caused by dynamically knocking elements out of the DOM. I fixed the problem and it seems that the FF incompatibility was also being caused by the same problem.

So far, there have been no more bug reports. Hopefully it will stay that way and I can get some peace!

On a side-note, Ive started work on a "resource site" by a group called Project Apex. Its a pretty nasty little organization. The heads of the group are highly experienced web users, some including hackers, programmers, scripters, and other ranks from the "dark net". They all agree that there is no reason to fear them or not visit their site as their objective isnt to mess with innocent computers. Check it out at http://www.project-apex.com.

Finished:Warn Logger

2006-03-04T20:22:00.000-06:00

Thats another code down! The long-awaited code Warn Logger is finally completed!

Check out this topic for details about how to install it. ^_^

Cookies Forever

2005-12-01T15:06:00.000-06:00

I thought of something yesterday when I decided to clear out my cookies and temporary internet files. When you clear out your cookies, all the information saved by codes goes bye bye, including all the statistics saved by non-php RPG hacks and other codes you have to work for to get the information you do! So, I sat there thinking through all the alternatives to "temporary cookies". The most sensefull of them was to have a text file on your machine where a program can save important cookie information and restore it every time you delete your cookies. The hard part shouldnt be getting the actual program made, but getting IF users to actually download and set it up on their computers.

Also for those of you interested in progress of Warn Logger, its almost done. Now that all my side projects are done, I can get back to mainstream coding for IF. :)

New: JS++

2005-10-16T18:49:00.000-05:00

While working on someones IF board, I realized how cool it would be if you could allow your advanced members to create their own JavaScript codes to run client-side everytime they visit the board!

I think it would be a pretty simple code structure. This project will become a reality after my other codes, such as Reply-Blocker update and Warn Logger, are completed. JS++ will exist in the User CP as a page containing a textbox for data entry. When the code in the textbox is submitted, a cookie will be created containing the data. Each page loaded past that point will have the cookie data appended to it. Since the inputted code will only affect the member who typed it up, there shouldnt be very many problems.

Thinking through the various functions and methods in JavaScript, I couldnt think of anything that would have to be restricted in this code. If you know of any variables, methods, functions, or other things that might cause a security or code flow problem please let me know by becoming a member of the UCO board and posting a reply to this topic. Any help is greatly appreciated.

Other Programs

2005-09-24T22:30:00.000-05:00

Ive started work on a collection of Habbo-related programs called Stalk Utilities. Most of the tools are done and are working correctly. The tools will be downloadable from Myishi Chat Center when the project is totally completed.

Also, Im getting to work on a program that allows you to read through emails from Dodgeit.com without having to know how to read HTML or XML. More on this program will be posted later.

As for Warn Logger, the code is starting to finally take form. Expect to see it sometime in October.

News

2005-08-18T15:25:00.000-05:00

A week or two back, I switched hosts for my codes. This was because my old host, geocities, had decided to kill my bandwidth. Even to today they havent given me more! So, I decided if they were going to be that stubborn I would just move to a different host. After moving all my public codes to webpost, a friend told me how their hosting is terrifically crap.

Now, Im looking for a better, more reliable host. I think Im just going to buy a site and move my devlog, programs, forum, codes, and all my other web content to it. But, we'll see! I really dont want to get a new host right after switching around. I might lose users that way.

In other news, today I took up a challenge from my computer instructor. He said that if I could create a FPS game with network capabilities and auto-patchwork systems, I would never have to do any work again all year. I can easily do it. That I have no doubt of. But Im concerned about how long it will take. Im guessing...about 3 weeks. If he looses interest or forgets about the challenge after Im done, I might be up a toilet without a crap! :(

New Features

2005-08-08T15:13:00.000-05:00

The smiley code is done now. When the page is done loading, all textual smilies will be turned into graphic smilies. Along with this feature is a new logo at the top of the page next to the title and description. Also, Ive added a list of smilies on the sidebar to show what the smilies look like before and after. As I add new smilies to the list, they will be added to the code as well. ;)

Update: Warn Logger

2005-08-03T20:02:00.000-05:00

Ive been working with iFusion to get all the bugs worked out of Warn Logger. So far, we havent been able to get it working the way it should be. Today I decided to install some debug alerts to help me figure out where the problems are occurring. The results were pretty surprising. It turns out one of the biggest problems was the fact that the code wasnt being run when the page loads! Im not sure how to fix this or even why it is doing this but hopefully after that problem is out of the way the rest will be easy to fix.

Some Things

2005-07-30T18:47:00.000-05:00

Im starting on a smiley code for this blog so all textual smilies will be automatically changed to graphical smilies when the page loads. But, dont expect it to be done until Warn Logger v1 is completed.

Arg! I just found out another one of my hosts killed my account. Thats the second one this month! So far, geocities seems to be the most reliable host that I have ever used. If you are hosted by a site that lets you use PHP and gives you a MySQL database for free, please comment on this blog.

Ive also shut down my old devlog on MCC and sent all the readers of it to this blog. So, welcome everybody!

Update: Warn Logger

2005-07-30T18:38:00.000-05:00

Im almost done with Warn Logger. I ran into some compatibility problems which led me to change around some things in the flow document. The updated version has been put in the warnlogger.zip file.

Warn Logger

2005-07-29T02:41:00.000-05:00

Alrighty then! Lets get this blog/devlog started!

I have a bunch of codes just sitting in my WIP folder. Ive decided to just stop working on all of them at the same time and focus on a request I got about two weeks ago (found here). Speaking of which, I was working on it yesterday when I hit a snag with the mechanics of it. So I decided to throw the entire code out the window and totally recode it.

I typed up a quick flow document and PHP form value chart to keep my ideas on the dot during the whole project. Both can be found in this zip file. In the href spreadsheet you can change the values in the generator table to show different data in the variable table. (That comes courtesy of a computer class I took last year ;))

Ill be typing up progress on this project as I work on it.

Post 1

2005-07-28T06:11:00.000-05:00

Hiya! My name is Blake and this here is my new programming blog! This will serve as my own little code publication system and Devlog. As I continue to code and advance in computer programming Ill add posts here to let you follow my work.

Like I said, this blog is pretty new. So please stick around while I fix the place up a bit. ^_^