Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
  • entries
    41
  • comments
    25
  • views
    5,077

About this blog

A random blog about technology related things. Updated whenever I have something on my mind.

Entries in this blog

 

Clever anti-piracy schemes

Many companies like to protect the sales of their software by incorporating anti-piracy schemes. A lot of them fall flat, being annoying and inconvenient to use to outright dangerous and perhaps legally dubious. However, there were a few that I thought were pretty clever in that not only was it transparent to people who purchased the game legitimately, but served as frustration to pirates at the same time.   Sony's "Wobble" Detection Mechanism On the PlayStation 1 Most of this is really from Technology Connection's video on the subject:   The gist of it is that as the laser is trying to read the disc, there are smarts in the reader in order to keep it on target with the data on disc. The way Sony did this was to use a couple of electromagnets in a flat X-Y plane configuration. The thing is that in the PlayStation, the reader also detected whenever the laser head moved around or "wobbled." In a particular part of the CD, an imperfection was pressed to induce a deliberate, specific wobble that the console understood. So if the wobble happened on startup, the console knew the disc was legitimate. Whenever a person tried to copy a PS1 game, the CD drive would read this data just fine. However, when it came to burning the disc, the burner wrote the data without the imperfections to induce the wobble. This meant that unless you had the actual CD press from Sony, you couldn't create a copy of a PS1 disc that the PS1 liked.   It was defeated by mod chips by pausing the disk after the initial verification was done, but certain games also were aware of mod chips installed on the system.   Insomniac's "Checuksumception" on Spyro: Year of the Dragon Insomniac faced a problem after the release of Spyro 2, it was heavily pirated. They didn't want this happening again with their new game and thought of a way to solve the problem. They came to one conclusion: You can't. But, what they did find out is that most of the time, a majority of sales of games happen within the first few months. So rather than try to stop pirates, they wanted to delay them from releasing a fully playable version of the game. Rather than try to do copy protection, the developers decided on crack protection.   The way they did this was sprinkled through the game were routines that calculated the checksum of the code as a means of integrity checking. But this wasn't enough. If a hacker can find where the routine to calculate the checksum is, they can just modify what's expected after making changes to the code. So Insonmiac decided another layer of checksum calculation was in order, using the checksum itself as part of the calculation. So effectively, checksums on checksums on checksums. And if you changed anything and tried to modify one of them, you'd have a ripple effect that would make all the rest invalid. Essentially, it was impractical at the time to modify anything in the code and still have the checksums line up.   But it didn't stop there. If the game detected an incorrect checksum was encountered, it would only break the game in subtle ways like removing collectibles used to advance to the next part of the game. This meant that when the hackers thought they broke the game's protections, they'd call it victory, release it, and then find out nope, they actually didn't. (from what it sounds like, people who make cracks are likely don't test it completely because in the world of making cracks, speed gets you cred)   The problem was that this method required 10 seconds or so of disc access. Obviously this can't do while you're playing the game so they limited it to just when the game is booting and masking it with showing logos and such. Another problem was this increased the development effort. But as a result of this method, it took over two months before the game was cracked in a time when a week was considered long. And the developers don't even think it was defeated, just bypassed.   Also if any of this sounds familiar to another DRM's method of protection, it's very likely that Denuvo runs on the same principle. Just using encryption instead.   You can read about the whole thing at https://www.gamasutra.com/view/feature/131439/keeping_the_pirates_at_bay.php   Breaking the Game If You Pirated It A few studios also took Insomniac's approach to piracy. Only they left out the "Checksumception" and used the "break the game" method. The ones that stood out to me over the years are: Earthbound's difficulty spike, getting booted to a PSA, and if you managed to get to the end, file erasure. The Immortal Pink Scorpion in Serious Sam 3: BFE Screwing up Batman's ability to glide in Batman: Arkham Asylum And my favorite because of its poetic justice, people pirating your game and causing your studio to go bankrupt in Game Dev Tycoon

M.Yurizaki

M.Yurizaki

 

[RQB] How the USB Type-C retention system works

Note: This is a copypasta of a status update I did.   One of the things that has eluded me about USB Type C was its retention system. On Micro B it's easy to see; those little bumps on the top of the cable's connector are it (they go into divots in the device's end). But in Type C I couldn't see anything. So I chalked it up to either friction or those six metal bits on the end of the cable's connector.   It turns out it doesn't look like either. I found an exploded diagram of the cable's connector (image from https://www.tweaktown.com/news/50182/wary-cheap-usb-type-cables-fry-hardware/index.html)   And I noticed there two claw looking arms colored dark teal. That looks like a retention system. So after digging around some more, I came to this website: https://www.appliancedesign.com/articles/94953-next-generation-connectors. They have a diagram of the assembly of the cable side's connector and sure enough, there are two claw type arms on either side of the connector. And those six metal bits may or may not be metal, but they look more for structural support than for retention.   So what about the device end?   There's two flanges out on the tab that has all the pins. And to verify this was really a thing, I poked at my phone's Type C port with a staple and managed to hook onto the flange.  So there you go, USB Type C's retention system.   (also I'm not sure if this was even discussed in the places where I first heard about the connector)

M.Yurizaki

M.Yurizaki

 

Why are physics engines tied to frame rates?

I came across the news about Fallout 76 has a problem: you can edit a value in one of the ini files which affects the frame rate, and then the physics engine is tied to that, with the end result is players are able to move faster simply by rendering more frames. Obviously this is a problem but why do developers design the physics engines around a frame rate? You need to have a rate of change. Frame rate is a convenient source of what the rate is.   A lot of equations in physics are over time. Meaning they measure something that happens between two points in time. Take velocity: it's the distance between two points in space divided by the time it took for something to travel that distance. Acceleration is the rate at which an object changes its velocity between two points in time. The problem with physics simulations in games (and perhaps in general) is that everything being calculated is being calculated for an instant of time. You might be able to know the input parameters, like how fast an object was going, but you won't know if the velocity will change in the future even if you knew all of the factors that are affecting it because you need one more thing: how long is the object going to be subjected to these effects? Or to put it in another way it's like asking this: I have an object going 5 meters per second, it's experiencing a acceleration of -1 m/s^2, what's its velocity going to be? I don't know, because I don't know how long this object will be experiencing said acceleration.   What Bethesda is doing though is they likely have a reasonably frequent enough rate at which the physics simulation is run at. But they also cap the frame rate to that rate as well. This may also cap the input sampling rate. So why would they do this? Because rendering faster than the rate at which the physics are simulated would mean rendering extra frames that don't add any new information. This may have been a design choice, because other developers don't seem to care like what Ubisoft does with Assassin's Creed: The cloth runs at "half frame rate" because the physics engine for this isn't ran as fast the graphics rendering is, and so you have this weird effect with the cloth seems to move at a disjointed rate.   So when you uncap the frame rate in a Bethesda game, you're effectively telling the game to run the physics engine at a faster rate, which is great!... except for the fact that if everything else about the game was designed around this one value and affects how everything else behaves. Really the solution to Bethesda's problem is to not make this value changeable in the first place. Or I guess, you know, design a physics engine that isn't dependent on frame rates.

M.Yurizaki

M.Yurizaki

 

My general software development axioms

An assortment of random axioms that I try to think about when doing software development work. The reason why I hold to these usually falls into one of these reasons: Making the most efficient use of my time Minimal need to maintain the code (hopefully, maybe) Ease of maintaining the code Ease of reading and understanding what the code does Finding the best way to make efficient software, without tramping on the above Regarding the software development process The priorities of development I stick to these priorities: Making the app easy to maintain If the app isn't easy to maintain, then it makes everything harder to do. And this is a very broad category, covering things like what tools I use, designing, and implementation details Making the app do what it's supposed to If the app doesn't do what it's supposed to do, then why bother using it? Making the app perform as best as it can Now some people who aren't developers looking into this may go "how come performance isn't your priority? Shouldn't you always make the app perform fast?" In particular say for games. To me, it doesn't matter if the game can get 300 FPS while looking good on potato hardware, if it crashes all the time, has bugs, or isn't, you know, fun, then I couldn't care less because it's not doing what it's supposed to do. And if it's a nightmare to maintain, then good luck fixing the problems.   Find the tools that you can work with, but only find as many as necessary Having an excellent set of tools does wonders for productive software development. The tools need to be ones that you can work with depending on your preference. But only get the ones that you need and no more. This creates noise in your toolchain and whenever you setup your development environment, the fewer tools you need to install and configure, the faster you can get to doing the actual work. And before you think about adding a tool to your toolbox in a more "permanent" fashion, think about why you need it, how has it helped you, and how often does it help you.   For example, the bare minimum toolset I would l like for a given project is: A text editor with code syntax highlighting and line number viewing on the side. Bonus points if it checks syntax and for issues, but that's not necessary A diff tool to see the differences between versions of source code and possibly merge them A version control tool of some sort. Some people may want more. Some people may want less. And some people use the bare minimum for these (if they do it). For example, The 8-Bit Guy for Planet X3 development uses a basic text editor (akin to Notepad) for coding and his "version control" is copying and pasting the project folder once in a while. He doesn't mention a diff tool, but he likely hasn't had a need for one.   Regarding software design Figure out what needs to be done first The app has a purpose, figure out what that purpose is and what needs to be done to achieve it first. If you don't know what the purpose of the app is, then what's the point in cranking out code for it?   Design from the top-down, but build from the bottom-up Do you design a skyscraper by first figuring out what kind of foundation you want before working on the shape you want it to have or do you design the skyscraper from the shape that you want it to have, then work your way down to the foundation? Or to put it more closer to geekiness, if you want to build a smartphone, do you first design the phone, its features, and what you'd like it to do, then work your way down to what kind of hardware it should have, or do you decide what hardware it should have, then work on the design of the phone, its features, etc.?   Granted you could go either way, but the problem with designing from the bottom-up is the bottom houses details aren't oriented to solving the problem of designing the thing in question. Those details are oriented to implementing it. Using the phone example, if you find out too early the hardware doesn't actually work the way you thought it did, then you'll be at square one. If you design from the top-down first, you end up with a concept that is independent of the implementation so that if you find out the hardware doesn't actually work, then the amount of work thrown away is relatively smaller.   So where does the "build from the bottom-up" come from? In a lot of cases, applications need a foundation and infrastructure to work with. By building from the bottom up, you create a platform in which makes developing the rest of the application easier. Using the smartphone example, once you're done designing and selecting the hardware, you need to first build the hardware, then, generally speaking, build the firmware, OS, and drivers before you can start working on the application itself.   Another way of saying this as well from say a web app point of view, build from the back-end to the front end.   Design principles are fine to learn, but should be understood before putting them to practice A design principle in software is always good to learn, but you shouldn't rush out and put it to practice before understanding what it's doing and how it would fit in your application.   Design (and coding) Principles The following principles I like to follow: Keep it simple, stupid (KISS): The simpler the pieces of code are, the easier it is to work with. Don't repeat yourself (DRY): If you are copying and pasting ANY amount, you should probably see if it fits better in its own function You ain't gonna need it (YAGNI): Don't do things you don't need until you actually need it. Loose coupling: Do not have one file, class, etc. heavily depend on another. That is, I should be able to remove this file, class, etc. without needing extensive modifications in other places of code. Single responsibility principle: A feature or a small subset of a feature should be contained within a single file, class, etc.   Regarding software implementation Code is written for humans, not machines It may be tempting to believe that the whole point of source code is to describe to a computer what you want it to do (and it'll do it exactly). However, that's not 100% true. Source code is describing what a computer should do in a way that humans know what the computer is doing.   Code should always be consistent, even when you don't agree with the style Everyone has their reasons for styling the code. But what's important is not the style of code, but that it's the same style across the board.   Code should self-documenting Every function, variable, class, etc. name needs to be meaningful. And not just meaningful by description, but context. For example all variables should be nouns, because they are a thing. All function names should start with a verb, because they are doing things.   Comments should be used sparingly and the only time I think it should be appropriate to use comments with no questions asked are: Separating sections of code Describing files at the header At the top of function definitions to describe them If something non-obvious needs to be communicated, like why an innocuous piece of code needs to be where it is Integrating someone else's solution is fine, if you understand what it does Let's face it, a lot of coding is basically copying and pasting someone else's code. But there's a difference in sticking a piece of code in your software, seeing it work, and going on your merry way than taking this piece of code, analyzing it, and seeing what it's doing, and how it affects your code.   Build and test often I always feel there's this stigma that if you have to constantly build and test out your application, you're probably not a good developer (This is probably self-inflicted, I never actually read anything about this). But to me, building and testing often means catching bugs and figuring out problems earlier than later. And when you change large swathes of code and there's a problem... well good luck finding what did it.   Work on one feature before moving onto another If the application has a lot of features or the work is updating a lot of features, focus on one feature before going to another. This will reduce mental load and allow concentrating on getting that one feature more or less perfected. If the feature is dependent on another one that isn't developed yet, provide dummy data or something.   Break things up if possible, but don't break them down too far Breaking things up helps understand the code piecemeal, but don't break it up so far that you're always jumping around in the code.   Reinventing the wheel is fine for simple things, but don't reinvent it again Basically, if you can hammer out a library or set of utilities for something simple, do it. And reuse it.   While I'm sure in Python there are CSV reading libraries, building one from scratch is easy and I'm not hunting one down to get the features I want. And when I do build it, I should keep it around so I'm not building it again.   Make the application predictable One of the biggest time savers for source code, both for the computer running it and the person reading it is making the application as predictable as possible. You can do this by trying to eliminate conditionals as much as possible. What I mean by a conditional is any if-statement or any loop. The less of these you have, the easier it is not only for the computer to process (predictable code is happy code), it's easier for developers to read and understand. To put another perspective on it: for every conditional you have, you double the complexity. Examples of eliminating conditionals: If you are initializing a variable and it has to be within certain bounds, find a way to make it impossible to go out of bounds in the first place. For example, if you are generating a random number and it needs to be within certain bounds, contain how the random number is generated in the first place so you don't have to check after the fact.
  Using a single variable to maintain state, rather than checking a bunch of variables to determine what to do next. If you catch yourself writing code that's always checking the same variables, it would be faster to have any change to those variables update a single "state" variable, then depending on what "state" its in, determine what to do next. Then when it comes to debugging a problem, you'll only have one thing to check: what was the last state it was in? And it's much easier to do a search on when the state changes to that particular value than say a dozen variables and finding the right combination that blows things up (I've had to deal with this before)
  Use switch-case statements when possible. If you need guidance on when you should use them, this answer on StackExchange helps. The compiler or interpreter will then find the best way to turn this into an cleanly executable bit of code https://www.eventhelix.com/RealtimeMantra/Basics/CToAssemblyTranslation3.htm is an example of how switch-case statements are handled in C.
  Avoid if-statements in loops if possible. An example of this is if you need to print out values, say the prime factorization of a number. This requires printing out an * after each value, but you don't want to print an * at the end of the last one. You could, in the loop that does the factorization, check if the value is going to be the last one and if it's false, print out an *. Or you can solve the first factor then in the loop, always print a * first thing. It boils down to this: make your code predictable. Deterministic. That is for any input, strive to make it run the same exact code.   Don't prematurely optimize Premature optimization is trying to be "smart" with code for the sake of performance at the cost of readability. Premature optimization also just adds additional work. You shouldn't be caring about the performance unless it's too slow. And if you need to care about performance, benchmark and profile the application to see where the heaviest cost is, then work on optimizing that. This reduces the amount of work you need to do.   It's okay to do the naive thing to prove the design works This ties in to "Don't prematurely optimize", but if you want to make sure the design works, do the naive solution just to prove what you're doing works. Once it does work, give it a once over for improvements if it doesn't take much effort.

M.Yurizaki

M.Yurizaki

 

[Random quick blog] What does Installing/Uninstalling a application really mean?

When you install an application using some install process, you'd think a lot is going on in the background aside from copying files to the install location. But for the most part, that's all it's really doing. Depending on how the application was designed and programmed, it'll be looking for files or libraries in certain places. So if said files or libraries don't already exist, the installer is making sure they get put in the right place. The only other thing that happens is to write some entry in the application manager that this application exists how to uninstall it. All uninstalling then is it's the reverse of this process. However, the uninstaller may not remove everything the installer put on, because some of those files may be libraries that other applications might use. An example of this is if the installer put on a version of Microsoft's Visual C++ runtime. The actual app uninstaller won't remove that because it's a library.   On that note, if all of the support files for the application are in place and anything extra it needs comes with the application, then there's really no need to "install" it. The application won't care it's not in the application manager nor will the OS (unless it's locked down for some reason).
 

Why multi-video card setups can't combine VRAM

( I need a name for blog posts like these, but all the good ones are taken )   While I don't think it's often brought up, an idea might come about that when using multiple video cards, such as in SLI or Crossfire, their VRAM combines. So if you have two 8GB cards, you effectively get the same thing as a 16GB card. However, this isn't the case. You might be asking... but why? If these setups combine GPU power, how come VRAM doesn't combine?   On a broader view, the video cards are given the same data so that they can work on the same thing. After all, they're generating frames from the same scene in the application. But wouldn't it be cool if you weren't limited to just the amount of VRAM on the card and expand beyond it? There's just a few problems with it: How is the data going to be transferred? If we look at PCI Express, it's a relatively slow interface compared to VRAM. PCIe 3.0 x16 caps at about 15.75 GB/s. NVIDIA's Titan V's VRAM has a bandwidth of a mind boggling 652 GB/s (imagine having that for your internet speed). So transferring data to and from cards would be an incredibly slow affair that would introduce stalls. To put in perspective, this speed difference is larger than that between SATA SSDs and DDR4-2133 VRAM works basically like a huge RAID-0 system. That is, each chip only has a fraction of the bandwidth of the card and it's the combined total of all of the chips performing that gives the bandwidth. So in order to transfer the data as fast as necessary to other cards, you would need a huge number of lines. I don't think connecting say 200 pin cables would be fun (nor would manufacturing them) Data transfers would have to be over a parallel bus. I've talked about this in some detail why high-speed parallel buses for usage outside of relatively short ranges stopped being a thing. But aside from the bulky cabling, there's also the issue of signal timing. It's going to be very hard to ensure that all the bits of a 7GHz signal will reach its destination at the same time, even if it's only say six inches end to end. A similar issue exists in systems with multiple physical processors. Though in this case, since all of the interconnects are on the motherboard itself, there's little issue with either making a huge cable or signal propagation. However, even in such cases, the system has to be aware of how to schedule tasks. As there's still a significant amount of latency accessing another processor's memory, some tasks will perform worse if scheduling considerations aren't taken into account.
 

When something seemingly designed well still has a problem.

With yet another security bug found on processors, one has to think how anyone would've let this through for this long. People would like to think there's incompetent engineering out there and while sure, they exist, what people also don't see are the designs that even you would agree with all of the knowledge and experience in the world that seems sound without experiencing it in the real world. So I have an example of such. This one I love to share, partly because pride (I was a junior developer who found a bug in senior developer designed code, showing that even people with 5-10 years experience can make mistakes), partly because this illustrates the point well.   A description of the system I was working on a system that comprised of a main controller unit and several wireless sensors. We had a rule with wireless communication in that we had to assume it's unreliable, even if 99.9999% of the time it appears reliable. This required that if a device transmits something, the recipient had to acknowledge it, or send an ACK. If the transmitter doesn't receive an ACK within some time, it'll retry sending the message. If three retries happened, then the device gives up on sending the message.   To handle this in software, we used a state machine. I forget the exact details, but this is what it looks like more or less on the transmitter side. This particular style of state machine is called a hierarchical state machine. The lines with arrows represent state transition with the event that triggers it. The default state is "Tx Idle" When it gets a "send message" trigger, it transitions to the "Tx Busy" state. After the message is sent, it goes to the "Waiting for ACK" state. This is a sub-state of the "Tx Busy" state because until the last request has been ACK'd, the transmitter won't transmit another message. If another message request comes in while in the "Tx Busy" state or its substates, it gets queued. If an ACK wasn't received in time, it moves up to the "Tx Busy" state again as the message is sent again. If an ACK was received or the message was retried enough times, it goes back to the "Tx Idle" state. If the system needs to send an ACK for any reason, the system immediately moves to the "Tx Busy" state. I forget the exact detail of this mechanism, but sending an ACK basically had priority over anything else this thing was doing. A buffer was included to queue up any messages (except ACKs) that needed to be sent if the hardware transmitter was busy sending something.   The problem: The message queue gets too full and breaks So the problem started when a project manager working on the system with us was doing a random test on his own. The system had 8 nodes that needed to transmit and receive data back and forth between a main unit. He invoked all of the nodes causing them to flood the main unit with messages that needed to be handled. If he did this long enough, the system would basically stop and "hang." There was a queue for requests in the state machine and if another one comes in but the queue was full, it'd trigger this behavior. Not that it was bug (i.e., hitting some overflow case), it basically failed an assertion check   My investigation led to the cause being that the number of requests coming into the transmission queue was outpacing how fast this state machine could go through it.   While I'll go over what happened, I want you to think about what the solution would be. You don't have to make a comment but stew on it. Just so you're not going blind, here are the parameters you'll be working with: The hardware this ran on at the time was an OMAP 3430. For those that don't know their SoCs, this was the same one that powered the Motorola Droid The devices connect through a ZigBee wireless network system. Unlike say Wi-Fi, ZigBee uses a mesh topology. This allows a device to only send data to the closest one, which will then send it to the next closest one until the ZigBee coordinator (the equivalent of a router in Wi-Fi) is within range. The ZigBee coordinator is within the main controller unit and communicates to the main board over a serial line at 115200 baud (or about 115.2 kbps) The messages were at most 300 or so bytes in length. Retry time is 100 milliseconds. At the time this problem happens, the system appears more or less fine (i.e., retries aren't piling up)       The root cause: There's an issue with the ACKing system The problem lies with the priority need for ACKs to be transmitted. The reason for having a "Tx Busy" state in the first place is not really as a courtesy, but that the serial lines are asynchronous. That is, once we fed the serial line some data and how much of it there is, it'll take care of the rest and the application is free to do other things. The state machine is waiting for the serial line to say "Okay, I'm done" before moving to the next state. However, whenever a "send ACK" request comes it, it gets sent regardless of what's going on.   Because of the way ACKs are short cutting the process, they are constantly keeping the serial line busy. This unintentionally can introduce a stall in the state machine where it never gets to the "Waiting for ACK" state. Or rather it gets there, but it's constantly pulled away from it. To put in another way, let's say you need to talk to someone, but there are other people who have higher priority than you who are allowed to interrupt you whenever. So whenever one of these higher priority people come in, they butt you out, speak to the person, and leave. But there's a ton of these people, and eventually your request never gets served (and you'll feel like punching one of these higher priority people).   (Note: I don't recall the exact way the serial line behavior was on the main unit, so there's some holes in the explanation here that I can't answer)   The solution is to deffer all transmission requests until the transmission is completed. So now the state machine looks something like this:   The fun part was the original state machine was also used in a few other places where some sort of communication with another device was happening. As you can imagine, this fix had to be propagated to various other parts of the system. And not only that, but we already had documentation with these state machine diagrams and such, so those had to be updated.   So remember: just because something looks sound, doesn't mean it's bulletproof. If you want to critique a huge issue cropping up, you're free to do so as long as you understand most of the time, these things go overlooked because they're not readily visible.
 

About that Task Manager CPU utilization "being wrong" (and about idling)

Note: I posted this as a status update, but it got long enough that I wanted to preserve it as a blog.   This popped up in my news feed: Netflix's senior software architect says Windows' CPU utilization meter is wrong. He has some good points in that it's not measuring the time a thread is actually doing useful work, rather than waiting on something, like data from RAM. Which he also points out that there is a gap in performance between RAM and CPU, but that's been a known problem for the past 30+ years.   In any case, while I like what he presents, I don't think Task Manager's CPU utilization is wrong, just misleading. All Task Manager's CPU utilization graph is measuring is the percentage of time in the sampling period (usually 1 second) a logical processor spent running the System Idle process. Nothing more.   The other thing is Windows can't tell if a thread is doing useful work or not unless it somehow observes what the thread is doing. Except the problem there is that requires CPU time. So it has to interrupt a thread (not necessarily the one its observing) to do this. And how often do you observe what a thread is doing? Adding this feature just to get a more accurate representation of CPU utilization is likely going to decrease general performance due to overhead.   If anything, I think it should be the app's responsibility to go "hey OS, I can't do anything else at the moment, so I'm going to sleep" when it actually can't do any more useful work. But a lot of developers like to think their app is the most important app ever and that any time they get on the CPU is precious, so they'll use up all of the time slice they can get.   Backup: About the System Idle Process Just so people are informed about the System Idle Process (taken from https://en.wikipedia.org/wiki/System_Idle_Process): Now you might go "but why bother with an idle process at all?" There's a few reasons described in https://embeddedgurus.com/stack-overflow/2013/04/idling-along/. A few of these are: "Petting the watchdog", a watchdog is a hardware timer that if it overflows, resets the system. This is for reliability reasons in case the system hangs. This action resets the watchdog timer. Power saving: If the CPU is running the idle process, you know the CPU is doing nothing. Depending on how aggressive you want the power saving to be, you can have it sleep the CPU as soon as it runs the idle state or after other conditions have been met Debug/logging tasks (Not mentioned in the article, but in a comment): if your tasks are time sensitive, you may not want them to dump stuff for logging or debugging since those specific tasks may not be predictable in the amount of time they spend. So you shove it to the idle task.
 

The Software Enigma Machine blog Pt. 4: The GUI

Part 4 in the making of the Software Enigma Machine. This last part deals with the graphical interface.   The Outline A recap on the outline. Part 1 What is the Enigma Machine? Why did I choose to implement the Enigma machine? Before Programming Begins: Understanding the theory of the Enigma Machine Finding a programming environment Part 2 Programming the features Rotors Rotor housing Part 3 Plug Board Part 4 GUI If you'd like to look at the source code, it's uploaded on GitHub.   Programming the GUI With Visual Studio, designing the GUI was pretty easy. Originally it looked like this:     The rotors were chosen as numeric spinner to both be an input and an output. An issue I was facing was at first I wanted the rotor position control to update the rotors internally if the value changed. This worked fine, but when I wanted to change their positions, I wasn't sure if I actually modified them or not (the "Set rotors" button originally didn't exist). So to fix this, I use the value change event handler to mark if the value is different from the rotor's position, but not update the rotor positions themselves. The user has to press the "Set Rotors" button to set the rotors and clear the highlighting. That ensures the user knows the rotors on the GUI report the correct position.   The "Seed" spinner is to set the randomization. If this changes, it creates a new set of rotors, and the rotors are tied to the seed. This helps increase the entropy (or randomness) of the system. If I did my math right, in theory there should be 637,876,516,893,491,200 possible combinations between the rotor settings (456,976 possible combinations), their wiring (650 permutations), and the random seed (2^32)   This version of the GUI is the "simple" version, in that you feed something in the Input textbox and the app will spit out the "scrambled" message in the Output textbox. To "decode" a message, the scrambled output is fed in as the input and the rotors are set back to the original position.   I knew this wouldn't really be a good Enigma Machine app if it didn't have the keyboard, lights, and plug board so that was next. The final design ended up having a tabbed interface for switching between the simple and the full versions:     The top section is the lamps for the letters, the middle for buttons, and the bottom for the plug board. The plug board also has three buttons for shuffling the wiring (produces a random output every time without affecting the RNG seed), resets the wiring to the first randomized position, or clears the board so the letters map to themselves.   So the biggest problem I had with making this GUI was while I'm sure there's a way to programmatically add elements, I still placed them all piece by piece (well, with some copy and paste for good measure). And then I had to rename them all with useful sounding names. But here came the worst part: how do I handle all of the necessary event handlers? I needed a handler for all of the keyboard key presses and one when one of the plug board letter mapping changes.   In Visual Studio, if you double click on a GUI element, it creates a default event handler for you for the most common action that would happen. So if you do this to a button, it will create an event handler for clicking on the button. Sure, I could do this for every button, and I can do this for every one of the plug board drop down menus. But here's the problem, now I have 56 handlers to edit. And 26 each have basically the same code, just using a different parameter. This is a classic case of repeating myself which is against the DRY principle (Don't Repeat Yourself). The biggest reason why this is a problem is if I need to change what an event handler does, I have go back and change it for all of them. That's error prone and gets old fast.   There was also a related situation on how I would light up a lamp. I could have something like this: string outputLetter = machine.ScrambleText(Input); switch(outputLetter){ case 'A': ALampLabel.BackColor = Color.Yellow; break; case 'B': BLampLabel.BackColor = Color.Yellow; break; ... case 'Z': ZLampLabel.BackColor = Color.Yellow; break; } Despite that I'm manipulating a different object in each case, this is still an example of violating the DRY principle. What if I wanted the lamp to show a different color for a scrambled letter? Now I have to change 26 of them.   At this point I realize C# Lists don't care what data type they store. They can store a collection of any data type. A programmatic solution can be done by having a collection of Labels, Buttons, and ComboBox types: Create a new List of the appropriate class type Use the List.Add method to add the objects. The caveat is that they have to be added in the correct order. This is where smart naming of the objects come into play. Since the letter mapping I feed the rotor housing and plug board is the English alphabet, I start with whatever represents A, then B, etc. At the end, if the elements need to be initialized to something, I use an iterator over the List and add what I need. For the buttons and drop down menus, they needed an event handler to handle the a click and a value change respectively. Fortunately, the way they handle the event is the same. For example, key presses just need to do this: keyLetter = plugBoard.GetRewiredLetter(keyLetter); encodeKeyPress(keyLetter); But where does keyLetter come from? I found out that every GUI element object has its own name as a property. So even though in code the object for say the keyboard key A is called AKeyboardButton, if I invoke AKeyboardButton.Name, it gives me "AKeyboardButton." And since the way I named the elements are the same, their schema is such that I pluck the first letter out and get the letter the key represents. This changes the code to: string keyLetter = keyButton.Name.Substring(0, 1); keyLetter = plugBoard.GetRewiredLetter(keyLetter); encodeKeyPress(keyLetter); Oh, but where did keyButton come from? That was taken from a parameter passed in the event handler. Event handlers have a parameter called sender of a generic object type. Casting sender to the appropriate object type (in this case, Button) figuratively turns the data type from a generic object type to Button. Now I can use sender as if it were the button I pressed. So the code is now: Button keyButton = ((Button)sender); string keyLetter = keyButton.Name.Substring(0, 1); keyLetter = plugBoard.GetRewiredLetter(keyLetter); encodeKeyPress(keyLetter); Using this technique probably saved not only a ton of lines of code, but a ton of headache needing to go back to change up anything should I needed to update how an event handler works. And I've already updated the event handlers a few times.

M.Yurizaki

M.Yurizaki

 

The Software Enigma Machine blog Pt. 3: The Plug board

Part 3 in the making of the Software Enigma Machine. This time, talking about the plug board.   The Outline A recap on the outline. Part 1 What is the Enigma Machine? Why did I choose to implement the Enigma machine? Before Programming Begins: Understanding the theory of the Enigma Machine Finding a programming environment Part 2 Programming the features Rotors Rotor housing Part 3 Plug Board Part 4 GUI If you'd like to look at the source code, it's uploaded on GitHub.   The Plug Board At first I thought I could reuse the rotor code, but there was a snag with this: the user can change the mapping at will.  And not only that, but the mapping needs to be reflective. That is if inputting A results in outputting G, then inputting G results in outputting A. So the first data structure I thought of that could store this mapping is a dictionary, where the keys and values are string types. The dictionary would be initialized to take in the list of letters or characters and just have them map to themselves. public PlugBoard(int Seed, List<string> CharList) { mapping = new Dictionary<string, string>(); rng = new Random(Seed); foreach (string entry in CharList) mapping.Add(entry, entry); } The RNG seed is needed for the shuffle function, which I'll talk about later.   At first I thought the remapping would be easy: public void ChangeWiring(string Input, string NewOutput) { mapping[Input] = NewOutput; mapping[NewOutput] = Input; } But this presents a problem. Say for instance we have this mapping:
  A => G B => S G => A S => B   I want to map A to B. But taking that code as is, would actually result in this mapping.   A => B B => A G => A S => B Oops. I didn't account for the fact that A and B still had something connected to it. So after stewing on this for a while, maybe thinking of some clever swap trick, I decided on this: don't assume the user wanted to remap the other side of Input and NewOutput. That is, those letters should be "disconnected" first. In this case, G and S should be disconnected and mapped themselves, then we can map A and B together. So the code becomes this: public void ChangeWiring(string Input, string NewOutput) { string oldOutput = mapping[Input]; mapping[oldOutput] = oldOutput; oldOutput = mapping[NewOutput]; mapping[oldOutput] = oldOutput; mapping[Input] = NewOutput; mapping[NewOutput] = Input; } Another feature I wanted is to have the app shuffle the wiring since: The user may not want to manually input what combinations they want on the plugboard It allows the app to automatically rewire the board in the event of encoding a text string, rather than having the user key in all of the text. The first instinct way of doing this would be to randomly generate two numbers of which indexes to change the wiring. Except... this is a dictionary where the key is a string. So I can't use an integer. And since I can't assume what the list of keys looks like, I can't use this integer to generate a character. However, you can iterate through the keys using the foreach loop. So I combined the two ideas so to speak: Generate a random number that's between 0 and the number of entries in the dictionary. Use a foreach loop to iterate through the letters. A counter counts up for each iteration. When the counter equals the random number, use the letter picked from the iteration and break the foreach loop. Do this twice, and the first and second letters to be wired together are picked. It looks hokey, but it works: public void ShuffleWiring() { int entries = mapping.Count; Dictionary<string, string>.KeyCollection keys = mapping.Keys; for(int i = 0; i < entries; i++) { int randEntry = rng.Next(entries + 1); int counter = 0; string firstLetter = ""; string secondLetter = ""; foreach (string randomKey in keys) { firstLetter = randomKey; counter++; if (counter == randEntry) break; } randEntry = (rng.Next() % entries); counter = 0; foreach (string randomKey in keys) { secondLetter = randomKey; counter++; if (counter == randEntry) break; } ChangeWiring(firstLetter, secondLetter); } } The only other plug board behavior needed is to get the mapped letter. However, since the valid inputs of the board are arbitrary, this method makes sure the input is a key in the dictionary. If it is, then it returns the remapped letter. Otherwise it returns the input as-is. public string GetRewiredLetter(string Input) { Input = Input.ToUpper(); if (mapping.ContainsKey(Input)) return mapping[Input]; return Input; } > On to Part 4

M.Yurizaki

M.Yurizaki

 

The Software Enigma Machine blog Pt. 2: The Rotors

Part 2 in the making of the Software Enigma Machine   The Outline A recap on the outline.   Part 1 What is the Enigma Machine? Why did I choose to implement the Enigma machine? Before Programming Begins: Understanding the theory of the Enigma Machine Finding a programming environment Part 2 Programming the features Rotors Rotor housing Part 3 Plug Board Part 4 GUI If you'd like to look at the source code, it's uploaded on GitHub.   Programming the Features This is where the programming magic begins, and how I went about coding each component.   The Rotors As a recap, the physical properties of the rotors are that they have pins, in this case 26 for the letters of the alphabet, a pin on one side maps to a pin on the other, and the rotors need to be able to "advance." One thing, I didn't want the rotors to be fixed to scrambling letters. So the pin names are referred to by their number, not by any letter or other character. And to help describe the "input" or "output of the rotors, since technically both sides could be an input or an output, I refer to the "left side" or "right side" of the rotor. The right-most rotor is always rotor 1, and is the first and last rotor the signal travels through.   So this makes the pin mapping easy: it's a List of integers. A List in C# is accessed like an array, so the index of the List are the pins on the "right side", while the value of that element in the List are the pins on the "left side." So to get a right side to left side mapping: return mapping[Pin]; However, there's a problem, how do you get the left side to right side mapping? There could be another List that contains the opposite mapping, but Lists have a method, IndexOf, that finds the first index that has a value. While this might be a problem if we had mappings to the same pin, the mapping is always 1:1 and unique, so there's no danger of using this method and have it return something that isn't actually where a pin is mapped. So getting the left to right mapping is: return mapping.IndexOf(Pin); That takes care of the basic aspect of the rotors. But they still have to "advance" or have an offset such that the same letter gets a different pin each time its selected. Also, each rotor has to have a different offset, since they don't advance at the same time. At first I tried doing some math with the Pin and adding an offset, but nothing lined up. And to make matters worse, I started with four rotors and I wasn't even sure if the rotors were working. So this confused the heck out of me when it was wrong and I couldn't figure out why. So instead I opted for the rotors to create a temporary list that copies entries from the original but at an offset. private List<int> getOffsetList() { List<int> tempList = new List<int>(); for(int i = 0; i < mapping.Count; i++) { int position = (i + pinOffset) % maxPins; tempList.Add(mapping[position]); } return tempList; } This worked, so I continued work from there. However I wasn't quite satisfied with this solution. While my initial check-in of the code has this, I went back to change how the mapping with the offset is obtained. But since this method works, I used this as a reference point to develop the new one using what pin is used as the input and what the current offset is. The result is: public int GetPin_RtoL(int Pin) { Pin = Pin + pinOffset; Pin = Pin % maxPins; return mapping[Pin]; } public int GetPin_LtoR(int Pin) { Pin = mapping.IndexOf(Pin) + (maxPins - pinOffset); Pin = Pin % maxPins; return Pin; } So to explain how this works: Let's look at this rotor. The input is pin A, which is connected to rotor pin L-1. The output doesn't matter, but going through the loop, the rotor advances and becomes this:   Pin A is now connected to rotor pin R-2, which will output Pin L-4. But from the perspective of the rotor, it really looks like this:   So to account for this, the rotor offset is added to the input pin. Since Pin A would've connected to rotor pin R-1, adding the offset (1 position in this case) and doing a modulo operation makes Pin A map to rotor pin R-2.   Now, what about from the left to right mapping? To start, let's say the rotor has advanced three positions: The input is coming into pin L-1 which maps to pin R-3. But from the rotor's perspective it's:   The problem is that the IndexOf method will return Pin R-3. If taken at face value, the app will think R-3 is the real output, which in the non-offset way maps to pin C (confused yet?). So we have to take the output pin value and modify it so the app believes the correct ABCD pin was outputted (in this case Pin D). To modify this, take the reverse mapping as normal, add how many pins there are on the rotor, then subtract the offset. Then, take that value and perform a modulo by the number of pins to get the correct mapping.   EDIT: Realizing how confusing that explanation is, here's a diagram to visualize how to get the answer:   With the main logic out of the way, all that's left is to get what the current position (or offset) the rotor is in and a way to set it. Those are straight forward and easy.   The Rotor Housing The rotor housing, for lack of a better name, contains all of the rotors and handles the interface between the rest of the system and the rotors themselves. These interfaces are: The key press scrambler What position the rotors are in Set the position of the rotors The constructor for this class can set an arbitrary number of rotors, how many pins they have, the random seed to use (this is to control the RNG), and a List that maps what "key" is tied to which pin. The mapping is used to translate a string character to a number since the rotor class uses a number. The number the rotor spits out is used to figure out which character to use as the encoded text.   There are two private methods to run the key input through the rotors and another to advance the rotors. In either case, since the rotors are in a List and a List can be accessed like an array, a first cut approach could be to use a for-loop to loop through each rotor. So the code would've been something like this: string output = ""; for (int i = 0; i < InputText.Length; i++) { string letter = InputText[i].ToString(); int inputPin = charMapping.IndexOf(letter); int outputPin = rotors[rotor].GetPin_RtoL(InputPin);; if (inputPin > -1){ for (int rotor = 1; i < rotors.Count; rotor++){ outputPin = rotors[rotor].GetPin_RtoL(outputPin); } /* Reflector rotor */ outputPin = rotors[rotor].GetPin_RtoL(outputPin); for (int rotor = rotors.Count; i >= 0 ; rotor--){ outputPin = rotors[rotor].GetPin_LtoR(outputPin); } output += charMapping[outputPin]; } else { output += letter; } } Seems simple enough, but I didn't want o take this approach. If only because I didn't like the multiple for-loop usage. So instead I decided upon a recursive method that looks like: private int getRotorOutput(int InputPin, int RotorNum) { int outputPin = 0; if(RotorNum < rotors.Length - 1) { outputPin = getRotorOutput(rotors[RotorNum].GetPin_RtoL(InputPin), (RotorNum + 1)); outputPin = rotors[RotorNum].GetPin_LtoR(outputPin); } else { outputPin = rotors[RotorNum].GetPin_LtoR(InputPin); } return outputPin; } The method takes in an input pin and which rotor to start on. If the rotor number isn't the reflector (noted by being the "last rotor"), it calls the function again. However the input that gets fed is the output of the rotor mapping (going right to left). When the reflector is hit, it doesn't call the method again, and so it returns the output. When the method returns with the right side pin being used of the last rotor, gets the left to right mapping, and returns that.   Admittedly, the for-loops method is easier to understand, but I wanted to use a recursive function for this on the idea that the rotor count is arbitrary and having multiple loops didn't sound appealing. A similar thought process was done with the rotor advancing method. To advance each rotor properly (the right side advances every time, the next one advances when the right side makes a full cycle, the next one advances when the previous one makes a full cycle, etc.), a straightforward approach would be to do something like: bool advanced = false; rotors[0].advanceRotor(); if(rotors[0].GetPosition() == 0) advanced = true; for(int i = 1; i < rotors.Count; i++); if(advanced == true){ rotors[i].AdvanceRotor(); if(rotors[0].GetPosition() == 0) advanced = true; } else break; } The recursive method is: private void advanceRotor(int RotorNum) { if (RotorNum == rotors.Length - 1) return; rotors[RotorNum].AdvanceRotor(); if (rotors[RotorNum].GetPosition() == 0 && RotorNum < (rotors.Length-1)) { RotorNum++; advanceRotor(RotorNum); } } Which again, while the for-loop method is easier to read, it also feels a little more complicated. Part of the problem is that, aside from the rightmost one, the other rotors only advance when the previous one made a full revolution. So the for-loop method has checking to see if the rotors made a revolution, and if it did, mark a flag so it can advance the next one. The recursive method doesn't need the flag.   There is a flaw with the recursive method, like most other recursion methods: if there are too many rotors, the stack could blow up. But given the small amount of rotors likely used for this code, the chances of it happening are limited. Unless you really want a 1000 rotor Enigma Machine. Even then I don't think the stack would blow up unless you want a stupid number of rotors.   > On to part 3

M.Yurizaki

M.Yurizaki

 

The Software Enigma Machine blog Pt. 1

Since enough people seemed interested in a status I posted some time ago that I thought, hey, it might be a good idea to do a write up. It'll be short enough that it'll actually end!   The Outline This section is just to provide an outline of this series: Part 1 What is the Enigma Machine? Why did I choose to implement the Enigma machine? Before Programming Begins: Understanding the theory of the Enigma Machine Finding a programming environment Part 2 Programming the features Rotors Rotor housing Part 3 Plug Board Part 4 GUI If you'd like to look at the source code, it's uploaded on GitHub.   A bit of background: What is the Enigma Machine? During World War II, everyone had their way of encrypting and decrypting messages. The Germans used a modified commercial product that did this called the Enigma Machine. It has a key for every letter (in my case, 26 for the English alphabet). Each key was electrically wired up so that when it was pressed, an electrical connection was made through it, then to a few rotors where it would come out to another key, which would light up a bulb corresponding to that key. So if you pressed A, it would get scrambled into something like D. Since it was electrically connected, you could also do something like swap letters so that the input would get swapped or the output would get swapped. From Wikipedia, here's a basic diagram of how it works (Pressing A results in a D, though it would've resulted in an S if not for the plug board connection in item 8): Though it wouldn't do much good if pressing A always resulted in a D. To solve this, every time a key is pressed, a rotor advances. In this case, the right most one. And when that makes a full revolution, it advances the next one, and so on. This makes it so that pressing the same letter always results in a different one. A key element in the Enigma machine was the reflector element (item 6). This was required due to how the machine was designed, which introduced a flaw in that a letter cannot encode to itself.   To decode the message, you take the encoded message, set the rotors to same position as when the encoding started, then type in the encoded message. The machine will light up the decoded letter for each encoded one.   In short, the Enigma machine is a rotating substitution cipher. Each combination of rotor positions represents a different substitution cipher that it rotates through automatically. There was also another substitution cipher in the form of the plug board, but this was manually entered and static.   Why did I choose to implement the Enigma machine? I was thinking of some simple projects to do on the side. While nice complex projects can be fun, they can also be a pain and having something done feels more like a sense of an accomplishment than getting 50% there on a complicated project after dumping hours into it.   The Enigma Machine came up because I figured it'd be a relatively good challenge and yet it's simple in theory. So it shouldn't be hard to debug and verify it works. And also World War II is my favorite time period, with the Enigma having interesting stories around it.   Before Programming Begins Most successful programming projects, or just any real project, needs a lot of planning and research ahead of time before the work can really begin. Poorly researched aspects can cause hiccups in the future that may not make themselves apparent until a lot of work has already started.   Understanding the Theory of the Enigma Machine There isn't much point in trying to program something if you don't understand how the system works in general. So step one is to understand how the Enigma Machine works. The most interesting part was the rotors, which aren't really that complex: there are input pins and output pins, and the input pins wire to an arbitrarily picked output pin. So the overall requirements for this system to work is: The pin mapping of the rotors should be randomly generated. However, this should not be uncontrolled random generation. That is, the user needs to input the RNG seed. Otherwise, the usefulness of the program is only good for that instance of the program. Each new instance would have a different randomly generated set of rotors with no way of setting it to what another instance used. For each rotor, an "input" must map to an "output" such that if you give the rotor an "input" value, it returns an "output" value. But if you give it the "output" value, it returns the "input" value. That is: If I give the rotor 1 and it returns 3, if I give the rotor 3, it returns 1. Each time a rotor is used, depending on its position, it needs to "advance". That is the next time I use the rotor, the mapping needs to be offset for the same input. For example: If the letter "A" maps to pin 1 on the rotor, the next time I feed the letter "A" into the rotor, it goes to pin 2 instead. The plug board has a similar idea, in that it's also a substitution cipher that works in the same way as the rotors, namely it needs the second requirement. However, they behave differently as far as the system is concerned so they need different sets of logic to function. These are: The plug board does not automatically advance. i.e., the mapping doesn't change automatically. The mapping can change based on the user needs. The rotors are more or less permanently wired. The plug board, as the name implies, has plugs the user can connect wires into and disconnect said wires. The user-changeable mapping presents a problem, say we have a mapping of A -> B, which means B -> A, and another mapping of C -> D, which means D -> C. What if I want to wire A -> C? The question will be answered in the section where I talk about the plug board design. Then there's the path of how a key press turns into a lit lamp: A key press goes to the plug board to be remapped. By default, the letter maps to itself. The output of the plug board goes into the input if the first rotor. The output of the first rotor goes into the input of the second pin for pin (i.e., rotor 1 outputs pin 2, so it goes into pin 2 of rotor 2) Repeat for N-rotors The output of the last rotor goes into the reflector, which spits out a value to go back into the nth rotor's "output" side, this spits out something on the "input" side. The "input" of the rotor goes into the "output" pin of the previous rotor, and repeat until the "input" of Rotor 1 is given. The returned "input" goes into the plug board to be remapped. The output of the plug board then spits out the letter. The key pressing and lamps are merely inputs and outputs. They don't affect the theory of operation of how the Enigma Machine works.   Figuring out What Language/Environment to Use I wanted this to have a GUI, because eventually I would need to have that keyboard + "light" interface. And because there are components in the Enigma Machine that do independent things and there could be multiples of one of them, I felt a language that supported object-oriented design is best. So these were the requirements for the language and environment I wanted: What has good GUI framework support? What supports object-oriented design? What features of the language makes things like manipulating entries in data structures or creating multiple instances of an object easier? What environment is easy to setup? Out of what I do know: Python: Python has a few GUI libraries, but my experience with this is limited. And what experience I do have isn't quite pleasant (to be fair, it was only with Tcl). I'm aware there are frameworks to make this easier, but I didn't want to spend time looking and testing them out. C++: Similar issue as I have with Python: I'm aware GUI libraries exist, my experience with them are limited (at least from a fresh start), and I don't want to spend too much time learning this. C: While the state of each rotors can be separated and business logic can just take in the state of rotors to work with, this would be hard than I'd like. Plus the issue with finding a GUI framework, though it's likely any C code I'd write would be called from C++ GUIs. JavaScript/HTML: I could, in theory, make the Enigma with this. But I didn't consider it for no reason other than it didn't cross my mind. C#: I'm sure there's some way of writing and compiling C# in a bare bones manner, but I use Visual Studio for C# work. And having worked with Visual Studio and C# to quickly write tools for work, I decided to go with this. Yes I'm aware Visual Studio has C++ support, but I don't have much experience with it. At least for developing Windows Form apps. There's also another reason why I went with C#: I know  it has a lot of data structures that make my life easier, because they come with a bunch of methods that help do what I want. And it's not just the language that has nice features that I want that's included, but setting up a C# development environment is stupid easy. It's just download and install Visual Studio. If I spent a day just setting up my tools, the project would've likely be dead in the water.   I like to think of myself as a lazy developer. Not in that my code is lazily written (most of the time, anyway), but I don't want to spend all day doing something that feels like it should take 10 minutes. The tools exist to make my job easier. If the tool doesn't appear do that, it's a poor tool.
With the planning phase done, the next entry will go over the programming process and how things were built.   > On to part 2

M.Yurizaki

M.Yurizaki

 

Outpost 2 - A game that should be given a second chance

I recently had a bug of a niche side of the city building genre: colony building. But then I remember an old standby that I still think should be used as the yardstick for all colony building games. That game is Outpost 2: Divided Destiny.   What is Outpost 2?   Outpost 2 was a game developed by Dynamics and released by Sierra in 1997. It can be summarized as Sim City meets Command and Conquer. Well, loosely speaking.   The premise is that humans have left earth because an asteroid annihilates it. And in their time drifting, they reject planet after planet until they find one is close enough. In desperation due to running low on resources, they land and hope to start anew. So there's a colony building aspect.   But humans being humans have different ways of solving a problem of trying to survive. One side wants to terraform the planet. The other wants to adapt to it. And things get heated and point lasers at each other. There's the combat aspect of it.   What makes Outpost 2 great? The first part has to do with the colony building aspect. Depending on the scenario, you start off with a central command center that is the heart of your colony: if it goes, the colony goes. But from thereon, you build, expand, get resources, make babies, research! Speaking of resources, there are plenty to manage, but most of the time you're not monitoring all of them constantly. The ones are metals to build stuff, food to feed the colonists, power to power structures, the colonists themselves (broken up into children, workers, and scientists), and the one you'll mostly be keeping an eye out, colony morale. The neat thing is a lot of the time you don't really have to actively manage them. You get regular reports both with a voice over and "pinging" if something needs attention or something happened.   The colony building itself is done by you. You put down the structures, layout the infrastructure, and generally make the colony. Speaking of which, there's only one infrastructure to worry about: tubes. And perhaps bizarrely, this is only to connect to the command center. Power is transmitted wirelessly. Food magically appears to colonists. Refined metals just teleport.   Here's a snippet of a colony from a scenario I'm playing:   There's a middle ground between macromanaging and micromanaging. You don't really need to tend to any single building or unit. The only time you do is because you want to use the building's function. But at the same time, you may need to juggle resources. If there's not enough workers to run a building, you may consider "idling" one until you get another worker. Morale may be the only one you really have to look out for, because there are many ways to influence it. Some directly, some indirectly.   The other part of the game is the lore. A whole novella was written and included in the game. It's something you have to read though after the mission briefing, but a surprising amount of backstory was available from the get-go. The game also came with an online manual (which was really just a large help file) and even the description of the units and structures get a blurb about what life is like for the people trying to survive. I mean, here's an example for a Vehicle Factory: And this extends a bit further in game. I mentioned there's research, which was starting to become the next big feature in RTS games of the time with regards to the tech tree. But I don't recall an RTS around that time that went into a level of detail such as this:     I mean, most other games probably would've cut it off at the first paragraph. The second part though, makes the game feel more immersive. Instead of the what the research is providing, it's telling me the why. Why should I spend my valuable scientists on this research? And not only that, the level of science fiction used in this game could be closer to hard science fiction than not. Sure there are some implausible technologies thrown around like "cool fusion" and "boptronics" (a combination of biological, optical, and electrical gadgets), but take this for example: The outcome might be a bit of a stretch (they found a way to use conductive fluid to generate electricity due to the planet's shifting magnetosphere), but this research topic is a real thing: https://en.wikipedia.org/wiki/Magnetohydrodynamics   This sort of thing tickled my imagination way back when I found it. It still sort of does. Either way, the amount of detail that went into this game, for an RTS that didn't deal with historical points or whatnot, is amazing.   What isn't so great about Outpost 2? I may love this game, but that doesn't mean it's not without its faults.   A large part of it is RNG. This game is full of it. While you can control how much metal you have, how much power you generate, and how much food you can produce, anything involving the colonists are random. You can only influence them. This means that in the beginning, you'll be struggling to break even as you have to squeeze every bit of resources you can get. And what makes things worse, one resource, Scientists, do not automatically generate and require scientists to create (by way of a University... which needs a scientist). So you can doom your colony to failure.   In addition to the colonists, the planet the people settled on is active. It has its natural disasters. While most of the time they take place away from your colony, you still have a chance of say a tornado spawning in the middle of your colony and wrecking everything. Damage control will be extensive.   But hey, maybe this RNG is part of the charm. You are trying to survive after all. But if this is too much random chance, it might be best to skip this.   There's also the announcer. While it's helpful at times to give you periodic resource reports, later in the game it starts to get a bit too chatty. One thing it announces? Every natural disaster. You can also research early disaster warning systems... which the announcer also announces. Of course, you can turn it off and still get reports, but it also means you have to keep an eye out on the resources tab more often.   And lastly, but this is more of a preference, the game is slow paced. One complaint I saw in a review of it at the time is that the game is slow. Not that it ran slow, but it's slower than others. It can actually take a while for units to move a good distance of the map. But this was at a time when RTS games were compared to Command and Conquer and Warcraft.   You did mention a combat system... I did! But part of me feels it was tacked on as an afterthought to make it try to compete with other RTS games at the time. It's not that it's bad and there's justification for it. But at the same time it feels slightly out of place.   In any case, combat is mostly relegated to robot vehicles (can't have the colonists risking their lives now). They shoot things. There's pros and cons. There's also guard posts that shoot things but they can't move.   But wait, this is a "2". Where's "Outpost"? (Or why Outpost 2 never really took off) The original game had a similar idea: Earth is dead, so go find a planet and build a colony. But the problem is that the game was released lacking features that someone hyped, it was unpolished, and the worst sin that any game can commit: it was unstable. This left a bad taste in people's mouths. So when a sequel came out, not only was it facing the genre defining games that I mentioned, but people still looked at this as yet another soppy colony building game that couldn't pass muster.   How do I get a hold of this game? So far it's being sold at Amazon from third party sellers for a fairly decent price: https://www.amazon.com/Outpost-2-Divided-Destiny-PC/dp/B0006OFKOQ/   I'd say if you like colony building games, this is one you should at least try. I feel this sets the bar for other colony building games. If not just for the game mechanics, but also how much they put into the lore and story.    

M.Yurizaki

M.Yurizaki

 

Think beyond computer problem solutions than just knowing them

Fixing computers is always a satisfying thing. Especially after chugging away at it for hours or even days. And when you come across the solution, you tuck it away in your memory, notebook, or what have you so the next time the problem shows up, you can fix it again. But I don't think that's enough to really "master" the computer.   I'm having a feeling that a lot of people who are beyond beginners accumulate solutions to problems or at least know how to find them online and simply spout out the solution. While this is fine for minor problems, more involved issues tend also seem to get little more than a glance as to understanding why something is happening in the first place. I find this detrimental and growth stunting in some ways.   Since it came up recently, I saw yet another reply in a thread where the OP mentioned cloning drives and the reply was "just reinstall Windows, you'll have problems if you clone." The last time I had any care about trying to talk to these people, I was linked to an article to someone who cloned it and shared their nightmare story about things were broken, but they just reinstalled Windows and everything was fine. It makes me wonder... have these people actually cloned their OSes or are they just spouting rhetoric that someone else wrote? Because I'm looking at myself going "I've cloned a half dozen times, none of them had issues." And then there's the fact that cloning is a popular way for mass deployment of a system. It's simply much faster to copy the data over to a new drive than it is to go through the setup process.   Without poking at the solution presented, it breeds misinformation that spreads. I did do a somewhat haphazard test of cloning an OS and used it, and my research only led to one thing that might possibly be it and it's caused by something someone would likely do. But if the solution to cloning is "don't do it, reinstall Windows because cloning causes issues" and the person can't explain why, then I don't think whoever says that doesn't understand the nature of the problem or the solution. They don't have to go down the specific details like "oh because xxx driver is dumb and hard codes where it lives" or "yyy setting affects zzz thing that's dependent on aaa thing", but something that at least makes sense.   Why is this important? Because establishing a root cause is helpful to knowing if the same problem happens again and you try the same solution you always have but it doesn't work, you can try a solution that's related.   One example of this is when people report their RAM usage is very high, but when they add up the amount of RAM their apps are taking up, it's much less. Like Task Manager reports 15/16GB is used, but the apps only use up 3GB. People often go "you might have a virus" (possible) or "something is wrong with RAM" (also possible). However, when I see this problem, I ask OP "Can you check if your non-paged pool is high?" Why? At first when I looked into this issue, the one thing that popped up a lot was that the Killer NIC driver's were leaking, and this causes the non-paged pool to increase in use. And so far most of the time when someone reports this, they're using a Killer NIC, so a driver update usually solves their problem. The root cause might be "Killer NIC drivers leak memory, causing the non-paged pool to increase." But then I dug around with what the non-paged pool is: it's kernel space memory that can't be paged out. So it's possible the poorly written drivers of any sort can do this. So the solution in general isn't just "If you have a Killer NIC, update the driver", it's "update your drivers if you haven't, one of them may be leaking and the update may have fixed it". I also found out after some random testing that VMs usually use the kernel memory space, which may also cause a situation like this to happen.   In another example, I work for a company that does system critical software. That is, this software needs to be reliable as failing can mean serious injury or death to those operating the machinery that software controls or to others that happen to be around it. When a problem comes up and it's severe, we investigate the hell out of it to find root cause. We do not like solutions that work but we don't know the "why." We still implement and post the solution, but it's not very helpful if we don't know what caused it. Because if it happened once, it'll happen again, and the solution may not be a complete solution, it may just lessen the severity of the consequences. And if we know the root cause to that problem, it may start lighting bulbs in our heads that the issue may have led to other problems.   So the next time you come across a complicated problem and solve it, try to figure out why that problem happened in the first place. You never know, it may allow you to fix your computer without consulting Google.

M.Yurizaki

M.Yurizaki

 

Project Dolly: A look into cloning Windows (Conclusions)

It's earlier than I'd said I would report this but for reasons I'm choosing to wrap up this experiment.   In day-to-day usage, I still haven't ran into any problems. Granted I did not play any games on the laptop, but I did run 3DMark after the cloning. However supposedly people do have issues regardless if they game or not. I also may not have been exactly representative of the use case, since I didn't clone after say a year of use. Though I can't think of anything that would cause issues since the only thing that should grow barring the installation of other programs is the Users folder and possibly ProgramData.   There was one other problem I forgot to mention in my last entry since it slipped my mind: the hard drive was offline when I first booted into the SSD. This means Windows knows the drive existed, but it didn't mount it. i.e., it wasn't accessible. I found an article explaining why this was though: http://www.multibooters.com/tutorials/resolve-a-disk-signature-collision.html. Windows has what is called disk signatures, which originally was meant to help the OS configure a software RAID setup. The disk signatures are supposed to be unique, but sometimes cloning software will copy this over. When that happens, Windows will take the other drive(s) with the same signature offline. You can put them back online, however, Windows will reassign a new signature to that drive. However, this is only a problem for that disk. So if you planned on booting into it again, you'll have issues.   The article claims that cloning tools used to either give you a choice in the matter of keeping or assigning new disk signatures. But now they automatically assign new signatures and change the configurations so Windows doesn't freak out.   So yes, there is some grain of truth that if you clone, Windows will run into issues because it's expecting one set of disk signatures and you've changed them. However, this appears to be a combination of the cloning tool and what the user did. Like for example in my case, since Samsung's cloning tool copied the hard drive basically verbatim, if I accessed the SSD right after cloning the hard drive (which is very likely if the person wants to do a soft verification of the clone), it would've changed the disk signature and I would've had issues. But since I booted into the SSD instead, I changed nothing. I'd imagine since few people know about disk signatures and if this was the root cause of their problems, this is why they think the act of cloning itself causes issues.   I used to clone via creating a bootloader partition, cloning C:\ onto the new drive with Partition Wizard (or Magic, I forget which), then making a bootloader on the drive. So disk signatures weren't really an issue for me with this method. However this hasn't been working for some reason or another and I don't really have a reason to clone drives these days so I never figured out why.   My conclusions on the matter: Cloning is more or less a safe thing to do. You should use the tool from manufacturer of the drive you're cloning tool if they have one. If the manufacturer does not have a tool, get a program that is advertised to do so like Macrium Reflect or Arconis True Image. After cloning do not try to access the drive. Do a verification by booting into it. If you do not see the original drive after booting into the new one, do not try to access it until you're satisfied with the cloned drive.

M.Yurizaki

M.Yurizaki

 

Project Dolly: A look into cloning Windows (Preliminary Results)

A couple of days ago, I decided to examine if cloning Windows was as bad as everyone says it would be, in that you'll run in to problems and other issues. So I'm using my laptop as a test bed for this project to see if anything really happens if you clone Windows.     When I started this project, I installed Windows 10 Fall Creator's Update to the laptop's SSHD, then installed programs that I would've used in a normal setting, and imported the settings of two of them (Visual Studio Code and Firefox). I also ran a 3DMark test suite and used the laptop as I normally would outside of gaming for a day. Today, I did the cloning process, which is cloning the SSHD contents onto the SSD. Since I have a Samsung SSD, I used Samsung's Disk Migration utility.   So here's the immediate results of the project so far: When on the SSHD Power on to login screen was 18.8 seconds. Reboot time from when the screen blacked out to login time was 26.8 seconds, with a BIOS time of 5.8 seconds. Superfetch was enabled The drive was being indexed The partition was 4K aligned (this is important) When on the SSD: Boot time was about 12.28 to 12.98 seconds with a BIOS time of 5.8 seconds Superfetch was enabled The drive was being indexed The partition was 4K aligned Trim was enabled and Windows knew it was an SSD in disk defragmenter 3DMark and CrystalDiskMark runs showed no anomalies, indicating that no system software got buggered. So far the only "issue" was after booting into the SSD for the first time after the cloning, Task Manager appeared to take longer to load. I did also notice one other thing while I was on the SSHD, it's actually not that bad to use. Perhaps it already cached the important things that I use but it wasn't really that bad compared to using an SSD. Load times were barely longer if I noticed and the system was still running mostly buttery smooth. Heck, most of the time when I logged in, it was practically instant (i.e., no "loading" or spinning circle)   I've already set myself on the path of no return by cleaning the SSHD. From here until the end of the month or so I'll be monitoring for issues that crop up.

M.Yurizaki

M.Yurizaki

 

Project Dolly: A look into cloning Windows (Planning)

I'll admit something. If there's one topic I typically stay away from because it seems like everyone has a consensus on the subject, it's cloning Windows installations from an HDD to an SSD. The consensus is it's always better to just reinstall the OS. While there are some practical reasons, the most common ones I've heard to avoid cloning is because you'll encounter a plethora of issues. I've found this puzzling considering: I've cloned multiple times with no ill effect I don't understand how there could be system software issues if the only thing that's changed is the hard drive. That's like saying if you upgrade your video card, reinstall Windows otherwise it'll freak out that it has a new video card. And as far as I understand about modern computer architectures, storage drives are basically virtualized and abstracted to the point where a storage drive is a storage drive. Well, the software still has to know how to talk to it, but as far its concerned, from an operational point of view there is no difference between a Samsung SSD, a Hitachi HDD, or a CF card talking over a SATA converter.   So I'm going to do a cloning project with my secondary computer, my Dell Inspiron 15 7559. It has a Samsung EVO 850 M.2 SSD and a Seagate SSHD. The plan is going to be like so: Install Windows on the HDD Make note of storage related services that Windows enabled. Notably check to see if Superfetch is enabled and if the drive is marked for defrag scheduling Install all of the drivers and programs I normally would've used on it. If I have importable settings from my primary computer, like from Firefox, then I'll import those. I'll install a few games on it. It won't be used much for gaming though when/if I do game on it. Give it a day or so to "settle" after running a few tests. Use Samsung's software to clone the HDD install over to the SSD Verify the SSD boot works, then wipe the HDD Check to see if the following has changed Superfetch (should be disabled) Defrag schedule (drive should not be available or is unchecked) TRIM is enabled If the above has not changed, continue on anyway, but make note of it Use the laptop for a few weeks Will mostly be doing light loads and maybe the occasional gaming if it's not too hard on the poor 960M Will also monitor any issues that come up After that, check to see if the following has been set Superfetch (should be disabled) Defrag schedule (drive should not be available or is unchecked) TRIM is enabled Note that even if these aren't enabled, that doesn't really affect anything since you can enable these settings. I was thinking about working with the laptop with just the HDD for more time, like a few weeks, but I don't want to stretch this project out too much.  

M.Yurizaki

M.Yurizaki

 

On the complaint of "incremental CPU improvements"

A common complaint I see about Intel is because they didn't have much in the way of competition from AMD for several years, they were content with releasing each new generation of processors with only "incremental" performance updates. Incremental being about 10%-15%. I wondered if in the past, we were enjoying a period of great performance improvements and so I went to looking around for benchmarks of processors from around the mid 2000s to late 2000s/early 2010s.   I found out that both Intel and AMD were only offering what amounted to incremental IPC improvements. Or perhaps just incremental improvements in general. The only exception was from Pentium D to Core 2.   Giant list of CPU reviews over the years   AMD Athlon 64 reviews (Compare against Athlon XP) http://www.anandtech.com/show/1164 http://www.tomshardware.com/reviews/amd,685.html https://www.extremetech.com/computing/55510-review-athlon-64-3400 http://hexus.net/tech/reviews/cpu/625-amd-athlon64-fx-51/   Core 2 Conroe reviews (Compare against Pentium D) http://www.trustedreviews.com/reviews/intel-core-2-duo-conroe-e6400-e6600-e6700-x6800 http://www.anandtech.com/show/2045 http://www.guru3d.com/articles-pages/review-core-2-duo-e6600-e6700-x6800,1.html http://www.pcstats.com/articleview.cfm?articleID=2097   AMD Phenom reviews (Compare against Athlon 64) http://www.hardwarecanucks.com/forum/hardware-canucks-reviews/9218-amd-phenom-x4-9750-quad-core-cpu-review.html https://www.bit-tech.net/reviews/tech/cpus/amd_phenom_x4_9850_9750_and_9550_b3_cpus/1/ http://www.legitreviews.com/amd-phenom-9900-processor-review-spider-platform_597   Core 2 Penryn reviews (compare against Core 2 Duo Conroe) http://www.anandtech.com/show/2306 (this is more of a preview than a real review) http://www.overclockersclub.com/reviews/intel_e8400/ (Though it's a review for the C2D E8400, compare the C2Q Q9450 against the C2Q Q6600) https://www.bit-tech.net/reviews/tech/cpus/intel_core_2_duo_e8500_e8400_and_e8200/1/ http://www.phoronix.com/scan.php?page=article&item=intel_c2d_e8400&num=1   Core i5 Lynnfield reviews (compare against Core 2 Quad Penryn) The reason why i5 is used instead of the i7 is the i7 has HyperThreading. http://www.anandtech.com/show/2832 https://www.techspot.com/review/193-intel-core-i5-750/ (Note they have an overclocked score that might through you off) https://www.bit-tech.net/reviews/tech/cpus/intel-core-i5-and-i7-lynnfield-cpu-review/1/   AMD Phenom II reviews (Compare against AMD Phenom 9950 BE, but keep in mind of the 0.2GHz clock difference in favor of the Phenom II) http://www.anandtech.com/show/2702 http://www.tomshardware.com/reviews/phenom-ii-940,2114.html https://www.bit-tech.net/reviews/tech/cpus/amd-phenom-ii-x4-940-and-920-review/1/   Maybe the most damning find: This pattern may have been the norm all along I found an article from 2003 on Tom's Hardware benchmarking CPUs from the Pentium 100MHz all the way to then contemporary Pentium 4's. All of the graphs seemed to indicate that 10%-20% IPC increases were the norm across generations of each CPU. You can read the article (starting from the test setup page) at http://www.tomshardware.com/reviews/benchmark-marathon,590-22.html   However, there are noticeable performance jumps, notably: Unreal Tournament 2003 does not run at even 30 FPS on Intel Pentiums and AMD K6s. However the K6-III was released around the same time as the Celeron Mendocino and I can't figure out why it performs so much better. The Celeron Mendocino was the second attempt with the on-die L2 cache, but K6-III had on-die L2 cache as well. Mp3 Maker Platinum and Main Concept (an MPEG-2 encoder) have huge increases in time on processors at K6-III/Celeron Mendocino and before. I would point to the programs using SSE, but AMD Athlon Thunderbird performs just fine and it doesn't have SSE. Gathering all of this data, I think the contributors that offered the biggest improvements over the past 20 or so years has been: Better caching. On-die L2 saved the Celeron Mendocino. Better front-end improvements like what happened on the Athlon. Some more reading can be found at http://www.anandtech.com/show/355. This also massively helped Intel when going from Pentium 4/Pentium D to Core/Core 2. Improvements to FPU performance. This may explain AMD's performance oddities in some of the tests on older processors while Intel seemingly did not have issues. I think the biggest takeaway though is that clock speed has improved greatly in a short period of time, and for a while, it was the best way to get more performance out of a processor. The Pentium P54C at 100 MHz was released in October of 1994. It took Intel about 6 years later in March of 2000 to release the 1GHz Pentium III. This is a clock increase of about 150MHz per year or a 1.523 times improvement per year on average. In March 2000, the 1GHz AMD Athlon was released. Four years later, a 2GHz Athlon was released. This is 250 MHz per year on average, but only a 1.19 times improvement per year. Intel took about 18 months between the initial 1.3GHz Pentium 4 to the first 2.6GHz Pentium 4, which is the most impressive improvement at 866.667 MHz per year. But to get to 3.8GHz, the fastest yet, took about 3 years from the first 1.9GHz processor. This is about 633.333 MHz per year or 1.26 times more improvement on average.   And another thing to point out, in some cases like the Pentium III 500MHz vs. 1GHz may not sound like much today since 500MHz differences are common in processors, but back then it was still a 200% performance difference on the same architecture.

M.Yurizaki

M.Yurizaki

 

We should really stop chastising people who think PC building is "too hard"

The other day I found a Gamer's Nexus video in my subscription feed. It was Steve Burke criticizing a magazine's article on how PC building is "hard" and to prove it wasn't, would do a speed build.   Admittedly I didn't watch the whole video, nor did I read the article in question. But my overall takeaway is this: to all you people who build or built their machines, stop saying it's easy as if building a PC is like operating an elevator or using a phone.   Now ignoring the other aspects of building a PC that conveniently get ignored, like identifying your needs, planning your budget, researching parts, testing, and if needed, troubleshooting, of course it's easy to build a PC if you've done it before. But to a fresh newbie, it still may be a nervous experience that really isn't all that easy for them.   To put in another way, I think about the time I was leaning how to ride a motorcycle vs when I ride now. I was nervous about shifting gears around except up when accelerating and down when coming to a stop, I had trouble with the clutch, hills were a problem, going down even past 45 MPH scared me, lane splitting was something I dared not to do with moving traffic.   Now, all of that no longer matters. I can shift just by my gut feeling. I have competent control of the clutch. I can manage most hills (though I'm sure San Francisco will humble me in a second). Highway speeds feel normal. I even lane split in moving traffic (and I survived LA).   These are all more or less "natural" to me. I don't have to think about how to do the action, but merely when to do it. So riding a motorcycle is easy to me. I think it's easy. But it's not.   Granted building a PC isn't this complex, risky, or what have you, but I wanted to illustrate a point. I'm sure most of you who built their own PC consulted the internet a few dozen times throughout each step of the process. I'm sure you spent a lot of time mulling over what's good for you or what's not. Or maybe you didn't and just copied someone else's build (though I'd argue you're sort of cheating). I'm sure when something went wrong, you nearly needed a change of underwear or blew a few blood vessels.   If PC building were really easy for anyone, nobody would need to do or worry about any of this.

M.Yurizaki

M.Yurizaki

 

Web app development is actually a good way to learn multi-threading concepts

As a way to expand my skill set and give me something to do in my spare time on programming, I've taken up learning what is called full stack web development. In it's most high level description, that means dabbling in both the front-end (client app, web page, etc.) and back-end (server, database, etc.) of the development process. To ease going into this, because I didn't want to learn almost a half-dozen languages, I've stuck with Node.js and MongoDB, as both use ECMAscript. Essentially all that means is I have to learn JavaScript. This has been a key point in where I'm going with the title.   A full web application is made up of several independent pieces Pieces like the server, the database, the client, and others. These pieces are independent form each other, yet they still need to talk to each other and perform as a cohesive unit. If one of these breaks or isn't developed well, the whole application can have a bad time.   Multi-threading means asynchronicity Synchronicity in this means that everything runs in order as you might expect while reading the code. Asynchronicity means that some code can execute right away, some code will be skipped, and some code will be put to the side, only to be run later.   If there is something you run into a lot of in JavaScript, it's that a lot of things are asynchronous. You have a lot of things on a web page that are waiting for user input. However, the web site can't freeze itself or force the user to do things in a specific order, because this decreases the quality of the user experience. In a server, when you do a database request, you don't want this hanging the entire server because a database request can take a while. You want the server to go right back to serving other people's requests if the current one is going to take a while and can be put off to the side. In JavaScript, these are handled by mechanisms such as callbacks and Promises. For example, here's some client code I've written, requesting to the server to get a list of groups: var getInfo = function(){ // Making an HTTP GET request $http.get('/groups/getList?index=' + $scope.startIndex) .then( function(response){ if (response.data.status === 30000){ $scope.groupList = $scope.groupList.concat(response.data.groupList); $scope.startIndex += response.data.groupList.length; } else { console.log("There was an error getting groups. Code:" + response.data.status); } }, function(response){ console.log("There was an error getting groups. Code:" + response.data.status); }); // Script resumes from here after making the HTTP GET request console.log("I made a HTTP GET request!"); }; The first part has the client make an HTTP GET request to the server to a particular URL for getting a list of groups. Since the server isn't going to respond instantaneously, the client will put this action off to the side and continue executing. What happens then is the client will execute console.log("I made a HTTP GET request!"); immediately afterwards. Essentially, for the moment since the data isn't available, it skips everything from the HTTP request function to the comment about where the script resumes. When the server sends the data back to the client, a signal is generated to let the client know to go back and execute the code that was skipped.   This is a key point in multi-threaded programming. Things need to be asynchronous. The application needs to be able to put tasks that are waiting for something else to be cast to the side and signal the application when the result comes in.   People may be sharing resources and you need to manage them An example is on Wikipedia. What if two people are editing a page at the same time, but adding different things to the page? How do you handle them both saving data? This is a common issue with multi-threading programming. One way of handling it the server having a token that only one person can take. If the token was taken, then nobody else can touch the page until the token is released. Another solution is if someone submits their edit, then another does, you can respond "hey, someone made changes already."   Web application development gives you more control in the debugging process This part makes understanding how each bit interact with each other easier.   One of the hardest things about developing and debugging multi-threaded applications is that a lot of things happen on the OS and processor level. It can be hard to poke into those to figure out what's going on. Like when you want to pause the execution of the application in the source code, you're not quite sure which thread you're on. When you're doing web application development, each instance you are accessing the web page can be thought of as a process or a thread in the entire application. In this way, you can pause one instance's execution and know exactly which one you're poking at.   In another sense, you may be able to control aspects behind the scenes, such as latency and such. It's harder to see how a local application will behave with hiccups and such without clunky methods, but you can certainly add latency to your network or simulate it another way without adding debug code or whatnot.   It's practical and relatively easy to setup! One of the hardest problems with learning something about software development is what to do with it. I'm finding full stack web development practical to learn because it touches upon something we all use in our daily lives. So instead of trying to figure out a local application you can run that can be multithreaded, try developing a website server and a client to go with it. The flip side is it's very visible and you can see your results.   Now I say it's relatively easy to setup, but it still has a learning curve. In my case, at the barest minimum you need to learn HTML and JavaScript. CSS is highly recommended. Then again, multi-threaded programming isn't a beginner level concept either. But if you take an afternoon to study, you can certainly get a web server setup: https://www.tutorialspoint.com/nodejs/nodejs_express_framework.htm   My development setup at the moment is: VirtualBox with Lubunutu Node.js  with Express as the server framework. MongoDB for the database. This is a document-oriented database, as opposed to a relational database like SQL. Whatever text editor of choice you want (I use Atom).

M.Yurizaki

M.Yurizaki

 

Does Making Windows 10 Lean Do Anything For Gaming Performance?

There's been some talk about Microsoft implementing a "gaming mode" for Windows which should help improve the performance of games. Or at least, minimize the impact of the OS on gaming performance. What this means is up in the air. However I decided to take a stab at something that could be like it and create a lean build of Windows 10. That is, a lot of its components were either disabled, deleted, or otherwise no longer a factor.   The Setup I'm using my computer for this test, which has the following specs:   Core i7-6700 16GB of RAM running at DDR4-2133 EVGA GeForce GTX 1080 SC ACX 3.0 256GB Samsung 950 Pro and 1TB Samsung 850 EVO   I created a 128GB partition on my 850 EVO for this, as I wasn't going to mess with my current build. Then I installed Windows 10 Pro with all of the settings on by default using the Windows 10 build as of somewhere in December 2016. Then I only installed the following:   Intel's chipset drivers NVIDIA's drivers, and only the drivers. The drivers for the Realtek ALC1150 chip 3DMark Unigine Heaven 4.0 Steam   The following tests will be used:   3DMark Sky Diver (Default settings) Fire Strike Extreme (Default settings) TimeSpy (Default settings) Unigine Heaven Extreme preset, with the resolution bumped up to 2560x1440 Final Fantasy XIV Heavensward Benchmark Default settings on the "High Desktop" quality GTA V 2560x1440 resolution No MSAA All settings on their highest, except reflection MSAA which was not set All the advanced graphics setting were set to their highest. Only Pass 4 (the final one) will be used for brevity. Deus Ex: Mankind Divided 2560x1440 resolution No MSAA DX11 Very high preset F1 2016 2560x1440 resolution Ultra high preset All tests were run three times and their results averaged.
  The Vanilla Build Results While I forgot to get a screen cap of RAM usage and processes going on, here's what it's like on my current build: Memory Usage:   Processes:   And here are the results:     3D Mark           Score Graphics Physics Combined Sky Diver Run 1 35045 70351 11034 23368 Sky Diver Run 2 35046 71077 11003 22992 Sky Diver Run 3 34835 70469 10904 23218 Average 34975.33333 70632.33333 10980.33333 23192.66667           Fire Strike EX Run 1 9588 10643 11682 4765 Fire Strike EX Run 2 9567 10619 11632 4762 Fire Strike EX Run 3 9553 10595 11662 4756 Average 9569.333333 10619 11658.66667 4761           Time Spy Run 1 6702 7229 4627   Time Spy Run 2 6699 7275 4626   Time Spy Run 3 6719 7286 4664   Average 6706.666667 7263.333333 4639             Heaven           Average FPS Score Min FPS Max FPS Run 1 64.7 1629 29.8 143.8 Run 2 65.4 1648 8.5 144.9 Run 3 64.8 1633 28.8 143.7 Average 64.96666667 1636.666667 22.36666667 144.1333333           FFXIV Heavensward           Score Average FPS Total Load Time   Run 1 14591 112.85 18.64   Run 2 14506 112.336 18.773   Run 3 14549 112.808 18.927   Average 14548.66667 112.6646667 18.78             GTAV           Pass 4 Min Pass 4 Max Pass 4 Avg   Run 1 41.248425 153.773926 82.296112   Run 2 36.866974 156.470566 81.178818   Run 3 40.984291 145.447479 75.742538   Average 39.69989667 151.8973237 79.739156             Deus Ex Mankind Divided           Avg Min Max   Run 1 62.4 50.5 77.3   Run 2 62.1 50.5 76.6   Run 3 62.2 50.5 76.9   Average 62.23333333 50.5 76.93333333             F1 2016           Avg Min Max   Run 1 92.3871 73.214142 111.866035   Run 2 95.232292 79.891083 118.127655   Run 3 94.716011 79.444923 116.410423   Average 94.111801 77.516716 115.4680377     As a curiosity, I ran F1 2016 again, this time with its priority set to "Realtime". Though it did have better performance, it was only up by 4-5 FPS at most. The other games saw no change so I didn't bother with them.   Making Windows 10 Lean One of the first things I did was go to Control Panel\Programs\Programs and Features > Turn Windows features on or off. Then went and removed the following: IE11 Media features Microsoft Print to PDF Print and Document Services Remote Differential Compression API support SMB 1.0/CIFS File Sharing Support Windows PowerShell 2.0 Work Folders Client XPS Services XPS Viewer   After a reboot, I did the following: A pplied a registry hack that while it does nothing for performance, it reduces a lot of wait times for things so it gives the impression performance did improve. In Control Panel\System and Security\System > Advanced System Settings and in the Performance section, set it to "Adjust for best performance." I did not touch the page file, as disabling it may make things worse. In Control Panel\System and Security\Security and Maintenance and disabled SmartScreen. Disabled hibernate In Settings app, did a blanket disable on everything that I didn't need.   Now came the question of all those Universal Windows Apps (UWAs), especially the system ones that run in the background all the time. I went to find where they lived, and well, they're spread out quite a bit. The first set is in C:\Program Files\WindowsApps under a super hidden folder that you need to take ownership of before you can even access it. I also had to use WinDirStat to even see the folder. It won't show up in Explorer, even if you disable "Hide system protected files and folders."   The first thing I tried is selecting everything and deleting it, but half the folders wouldn't be deleted for some reason. Rather than try to fight it, I found out I could rename them. So I did:   Except this isn't where all of them live. Some live in C:\Windows\SystemApps. This is a doozy, because if they live here, they must be vital right? Well, good thing they really aren't. Most of them anyway. However, I didn't delete these, I just renamed them.   A few things to note here though: ContactSupport_cw5n1h2txyewy is for the Contact Support app Microsoft.MicrosoftEdge_8wekyb3d8bbwe is for Edge Microsoft.Windows.Cortana_cw5n1h2txyewy is for Cortana If you disable Cortana, you also lose Start Menu Search Microsoft.LockApp_cw5n1h2txyewy is for the lock screen ShellExperienceHost_cw5n1h2txyewy is for the Start Menu. Microsoft.XboxGameCallableUI_cw5n1h2txyewy is for the Xbox Live app   In here I disabled everything but the LockApp and Start Menu, because you can disable the lock screen in other ways and not having a Start Menu is odd. The task bar will still work though. Make sure that before you do this, unpin any apps from this list. Otherwise, they'll be in limbo where they're pinned.   And lastly, yes there's more, there's one more place to disable UWAs, otherwise they'll automatically run. These live in C:\Users\[username]\AppData\Local\Packages:   Now that the UWAs are out of the way, I went and looked for services and such to disable. To find which ones were safe to do so, I went to Black Viper's website. Anything that was safe to disable was disabled.   And just for kicks, I set the power profile to "High performance."   So What Kind of Performance Does This Get Me Now? After letting Windows settle for a bit, I got it down to this:     Shaved off nearly 600MB of RAM usage and got the process list down quite a bit. Note that the "Non-paged Pool" is half (~75MB) of what my current build is (~150MB). This is the core operating system components. So I must've done something good there! Right?   Enough of that, what kind of performance are we getting now?     3D Mark           Score Graphics Physics Combined Sky Diver Run 1 35089 70950 10993 23352 Sky Diver Run 2 35177 71106 11099 22911 Sky Diver Run 3 34987 70397 10983 23418 Average 35084.33333 70817.66667 11025 23227           Fire Strike EX Run 1 9566 10603 11697 4768 Fire Strike EX Run 2 9577 10634 11696 4750 Fire Strike EX Run 3 9567 10609 11645 4775 Average 9570 10615.33333 11679.33333 4764.333333           Time Spy Run 1 6705 7287 4618   Time Spy Run 2 6689 7267 4614   Time Spy Run 3 6715 7282 4661   Average 6703 7278.666667 4631             Heaven           Average FPS Score Min FPS Max FPS Run 1 66.2 1667 27.1 144.9 Run 2 66 1663 25 145.9 Run 3 66 1663 29.7 146.1 Average 66.06666667 1664.333333 27.26666667 145.6333333           FFXIV Heavensward           Score Average FPS Total Load Time   Run 1 14497 112.431 18.796   Run 2 14550 112.619 18.573   Run 3 14610 113.03 18.791   Average 14552.33333 112.6933333 18.72             GTAV           Pass 4 Min Pass 4 Max Pass 4 Avg   Run 1 20.244783 154.143906 81.087631   Run 2 39.342747 154.573441 82.079002   Run 3 20.863869 115.898499 81.30619   Average 26.817133 141.5386153 81.490941             Deus Ex Mankind Divided           Avg Min Max   Run 1 62.4 50.2 76.9   Run 2 62.1 50.5 76.9   Run 3 62.1 49.8 76.6   Average 62.2 50.16666667 76.8             F1 2016           Avg Min Max   Run 1 97.294609 80.587448 119.962158   Run 2 97.303444 81.302322 118.235237   Run 3 95.821739 80.525665 118.570518   Average 96.80659733 80.805145 118.9226377     Er... Almost no change at all. With all of these system resources freed up, wouldn't that mean performance should also go up? Well, not really. Most of what I disabled and turned off either wasn't actively being used to begin with, or they're mostly sitting around, waiting for something to happen, with maybe some background activity going on if it detects you're not doing something. i.e., the priority is lower than normal. And this makes sense. If the OS had components that were actively using the CPU time, then something is wrong. One of the OS's main job is to provide services to user applications when they need it. Those services should be running either on demand or at a very low rate.   But I do want to make a note on the user experience. I think it actually improved. Although I think that had something to do with the registry hack pack I applied and removing animations more so things just taking up too much resources. Boot times were the same, if not feeling a little worse (perhaps due to Windows trying to find applications whose folders I renamed). However, shutting down is practically instant.   Conclusions If you have a high-end machine, you can expect almost no performance improvement for making a "lean" Windows build. For lower end machines, it certainly might help, but I think what'll really help is just disabling a lot of the GUI fluff. Most of the applications Windows runs run in the background and are idling most of the time.   I didn't bother reading, I just want the results of Vanilla Vs "Lean" Windows   Vanilla Windows         Lean Windows         % Diff (Vanilla vs. Lean)       3D Mark                               Score Graphics Physics Combined   Score Graphics Physics Combined   Score Graphics Physics Combined Sky Diver Run 1 35045 70351 11034 23368   35089 70950 10993 23352   99.87460458 99.15574348 100.3729646 100.0685166 Sky Diver Run 2 35046 71077 11003 22992   35177 71106 11099 22911   99.62759758 99.95921582 99.13505721 100.353542 Sky Diver Run 3 34835 70469 10904 23218   34987 70397 10983 23418   99.56555292 100.1022771 99.28070655 99.1459561 Average 34975.33333 70632.33333 10980.33333 23192.66667   35084.33333 70817.66667 11025 23227   99.68925169 99.7390788 99.59624279 99.85600489                               Fire Strike EX Run 1 9588 10643 11682 4765   9566 10603 11697 4768   100.2299812 100.3772517 99.87176199 99.93708054 Fire Strike EX Run 2 9567 10619 11632 4762   9577 10634 11696 4750   99.89558317 99.85894301 99.45280438 100.2526316 Fire Strike EX Run 3 9553 10595 11662 4756   9567 10609 11645 4775   99.85366364 99.86803657 100.1459854 99.60209424 Average 9569.333333 10619 11658.66667 4761   9570 10615.33333 11679.33333 4764.333333   99.993076 100.0347438 99.82351726 99.93060212                               Time Spy Run 1 6702 7229 4627     6705 7287 4618     99.95525727 99.20406203 100.1948896   Time Spy Run 2 6699 7275 4626     6689 7267 4614     100.1494992 100.1100867 100.260078   Time Spy Run 3 6719 7286 4664     6715 7282 4661     100.0595681 100.05493 100.0643639   Average 6706.666667 7263.333333 4639     6703 7278.666667 4631     100.0547749 99.7896929 100.1731105                                 Heaven                               Average FPS Score Min FPS Max FPS   Average FPS Score Min FPS Max FPS   Average FPS Score Min FPS Max FPS Run 1 64.7 1629 29.8 143.8   66.2 1667 27.1 144.9   97.73413897 97.72045591 109.9630996 99.24085576 Run 2 65.4 1648 8.5 144.9   66 1663 25 145.9   99.09090909 99.09801563 34 99.31459904 Run 3 64.8 1633 28.8 143.7   66 1663 29.7 146.1   98.18181818 98.19603127 96.96969697 98.35728953 Average 64.96666667 1636.666667 22.36666667 144.1333333   66.06666667 1664.333333 27.26666667 145.6333333   98.33562208 98.3381676 80.3109322 98.97091478                               FFXIV Heavensward                               Score Average FPS Total Load Time     Score Average FPS Total Load Time     Score Average FPS Total Load Time   Run 1 14591 112.85 18.64     14497 112.431 18.796     100.64841 100.372673 99.17003618   Run 2 14506 112.336 18.773     14550 112.619 18.573     99.6975945 99.74871025 101.076832   Run 3 14549 112.808 18.927     14610 113.03 18.791     99.58247775 99.80359197 100.7237507   Average 14548.66667 112.6646667 18.78     14552.33333 112.6933333 18.72     99.97616076 99.97499175 100.3235396                                 GTAV                               Pass 4 Min Pass 4 Max Pass 4 Avg     Pass 4 Min Pass 4 Max Pass 4 Avg     Pass 4 Min Pass 4 Max Pass 4 Avg   Run 1 41.248425 153.773926 82.296112     20.244783 154.143906 81.087631     203.7484176 99.75997754 101.4903395   Run 2 36.866974 156.470566 81.178818     39.342747 154.573441 82.079002     93.70716793 101.2273292 98.90327127   Run 3 40.984291 145.447479 75.742538     20.863869 115.898499 81.30619     196.4366772 125.4955675 93.1571606   Average 39.69989667 151.8973237 79.739156     26.817133 141.5386153 81.490941     164.6307542 108.8276247 97.85025713                                 Deus Ex Mankind Divided                               Avg Min Max     Avg Min Max     Avg Min Max   Run 1 62.4 50.5 77.3     62.4 50.2 76.9     100 100.5976096 100.520156   Run 2 62.1 50.5 76.6     62.1 50.5 76.9     100 100 99.60988296   Run 3 62.2 50.5 76.9     62.1 49.8 76.6     100.1610306 101.4056225 100.3916449   Average 62.23333333 50.5 76.93333333     62.2 50.16666667 76.8     100.0536769 100.667744 100.1738946                                 F1 2016                               Avg Min Max     Avg Min Max     Avg Min Max   Run 1 92.3871 73.214142 111.866035     97.294609 80.587448 119.962158     94.95603194 90.85055281 93.25110257   Run 2 95.232292 79.891083 118.127655     97.303444 81.302322 118.235237     97.87145047 98.26420825 99.9090102   Run 3 94.716011 79.444923 116.410423     95.821739 80.525665 118.570518     98.84605726 98.65789124 98.17821914   Average 94.111801 77.516716 115.4680377     96.80659733 80.805145 118.9226377     97.22451322 95.92421743 97.11277731    

M.Yurizaki

M.Yurizaki

 

A trip to The Computer History Museum

Since I was in the area, I decided to take a trip down to the Computer History Museum in Sunnyvale, California. I also could've made a pit stop by Google's HQ and Apple's HQ for maximum geekiness (well and maybe glance at Oracle and Adobe's HQ), but nah. It's a nifty place if you get the chance to go. Just don't expect much out of the gift shop, there's a lot of cringe-worthy "geeky" stuff. But then again, maybe you're into that sort of thing :3   Since a picture dump really isn't feasible, I thought I'd share some images that sparked fun thoughts in my mind.   How's this for your computer's battlestation? (Control station for the UNIVAC)   Programmers, remember to give respect where due!   Light gun tech, in the 50s. Makes me wonder if they did it because they knew the station operators would be bored out of their minds and pretending to "shoot down" bogeys would be fun.   It's the Apollo Guidance Computer. Most of it if not all of it. This thing's an impressive piece of work.   Check out that SSD from 1990 (top right)   I'd probably vote for him.   "Stealing" software was a big thing up until the 80s apparently. But then again, the idea that a third party could write software for a multitude of systems wasn't really heard of.   Imagine if this were your CPU. o_O (Also 15 GFLOPS would've been attainable by CPUs within 10 years and GPUS within 8 or so)   Here's a Cray-1. It peaked at 160MFLOPS. To put that in perspective in terms of today's technology, a GeForce GTX 1080 is about 63,480 times more powerful. Okay, smartphones in 2013 were 10 times more powerful. It only took about 40 years to get there =3   "We got these SNES carts, what do we do with them?"   Check out that Pentium Pro in the right of the center row. The bottom right is a Pentium 4, which is about the size of every Intel desktop processor since.   It's the Google car! It was okay.   This is an analog computer. You think your computer has bad cable management?   Hm, I wonder if AMD got the name "ACE" from this...

M.Yurizaki

M.Yurizaki

 

Let's build the PS4 Pro

EDIT: It's probably a good idea to lay out some ground rules at the top so people know how I'm approaching this: Everything must be new. The reason being the used market is not consistent or reliable. Everything must be gotten from official channels. Hardware must be matched as closely as possible. If the exact part or equivalent cannot be found, a justification must be needed. If a part on the PS4 is available for the PC, it must be included. If I'm allowed to leave out some parts though, its price must be reflected against the console. So if I omit a $40 part, I must subtract $40 from the PS4's price. Certain things that are not known like the secondary ARM chip and its DRAM, can be left out for the sake of argument. Every once in a while I like to see if I can build a machine that closely matches the PS4's specs as much as possible and see how much it would cost. I've done the PS4 a few times, so let's do the PS4 Pro:   PCPartPicker part list: https://pcpartpicker.com/list/g9L6pb
Price breakdown by merchant: https://pcpartpicker.com/list/g9L6pb/by_merchant/ CPU: AMD Athlon X4 880K 4.0GHz Quad-Core Processor  ($92.34 @ Newegg Marketplace)
Motherboard: ASRock FM2A68M-DG3+ Micro ATX FM2+ Motherboard  ($44.99 @ SuperBiiz)
Memory: Crucial 4GB (1 x 4GB) DDR3-1600 Memory  ($24.88 @ OutletPC)
Storage: Toshiba 1TB 2.5" 5400RPM Internal Hard Drive  ($54.89 @ SuperBiiz)
Video Card: Gigabyte Radeon RX 480 4GB G1 Gaming Video Card  ($189.99 @ Newegg)
Case: Rosewill RANGER-M MicroATX Mini Tower Case  ($24.99 @ Newegg)
Power Supply: SeaSonic S12II 430W 80+ Bronze Certified ATX Power Supply  ($49.99 @ Newegg)
Optical Drive: LG WH14NS40 Blu-Ray/DVD/CD Writer  ($46.88 @ OutletPC)
Total: $528.95
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2016-12-09 13:02 EST-0500   Most of the parts were the cheapest parts I could source that at least wouldn't blow up in my face (hence the SeaSonic PSU). But let's go over the key points here: Processor: The PS4 Pro has a 2.1GHz 8-core Jaguar. In this I included a quad-core Athlon X4 running at 4.0GHz. The assumption is that doubling the speed but halving the cores will result in similar performance. Also the Athlon X4 uses the Steamroller architecture, which should be a bit better than Jaguar's GPU: The PS4 Pro has something that resembles a downclocked RX 480 (911MHz vs 1120 MHz). So I went with the 4GB version of the RX 480 mostly because from the only resource about GPU memory consumption I could find on the PS4, graphics consumes about 3.5GB. Memory: 4GB for the GPU leaves 4GB for the system to keep it "fair". That same resource also suggested that 3 GB or so was used for the system outside of the game. Storage: The PS4 Pro comes with a 1TB 5400RPM 2.5" drive, and probably a cheap one too. Optical drive: Because the PS4 Pro has one and I need to keep this fair. But it is the cheapest one PCPP had listed. Hm... Even without an OS or peripherals I'm still about $130 over and there's not a whole lot I could cut off.   EDIT: As of 01/04/2017, the price jumped up another $36 for the parts, to a total of about $566   PCPartPicker part list: https://pcpartpicker.com/list/g9L6pb
Price breakdown by merchant: https://pcpartpicker.com/list/g9L6pb/by_merchant/ CPU: AMD Athlon X4 880K 4.0GHz Quad-Core Processor  ($92.29 @ SuperBiiz)
Motherboard: ASRock FM2A68M-DG3+ Micro ATX FM2+ Motherboard  ($44.99 @ SuperBiiz)
Memory: Crucial 4GB (1 x 4GB) DDR3-1600 Memory  ($23.48 @ Amazon)
Storage: Toshiba 1TB 2.5" 5400RPM Internal Hard Drive  ($54.89 @ SuperBiiz)
Video Card: Gigabyte Radeon RX 480 4GB G1 Gaming Video Card  ($224.98 @ Newegg)
Case: Rosewill RANGER-M MicroATX Mini Tower Case  ($33.30 @ Jet)
Power Supply: SeaSonic S12II 430W 80+ Bronze Certified ATX Power Supply  ($44.98 @ Newegg)
Optical Drive: LG WH14NS40 Blu-Ray/DVD/CD Writer  ($46.88 @ OutletPC)
Total: $565.79
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2017-01-04 19:11 EST-0500

M.Yurizaki

M.Yurizaki

 

How the computer world became serial

Practically every peripheral interface, be it USB, SATA, PCIe, Thunderbolt, and even DisplayPort, uses a serial interface. That is, it transmits one bit at a time across the data channel. That seems kind of inefficient, considering that we used to have parallel interfaces that could transmit many bits at the same time. In fact, the only truly parallel interface that remains is DDR memory and inter-processor communication. So what happened? To understand this, it's best to look at several aspects of getting computers to talk to each other.   How do devices talk to each other anyway? Through electrical signals of course! But these signals must be understood by all parties before those signals can make any sense. You might think this is as simple as sending a high or low voltage, but there are many characteristics of a signal to come into play here. These include: Voltage level: this is necessary for the devices to figure out what's a 1 and what's a 0. Some protocols don't have a 0V and some voltage. The official RS-232 spec for instance uses a positive and negative voltage. Baud (bd): this is the fastest the signal can swing between high and low states. This is somewhat interchangeable with bits per second, but technically, for example, 0000 is 0 baud while 1010 is 4 baud, even though both are 4 bps. With compression, the data rate can exceed the baud. Control: These are special things that happen within the signal that the receiver can use. For example, in the RS-232 spec, the transmitter sends a "start" bit that's a high value and ends with a stop bit that's a low value. Others can embed a clock signal by changing the signal between bits.   These three attributes define a protocol, or the "language" that the devices speak. Once this is agreed upon, the following happens: The receiver remains idle, keeping just enough things up to wait for some kind of "start" flag. In many protocols, an inactive line is 0V to avoid drawing power over the line while nothing is going on. The transmitter has a buffer where the next byte being sent out lives. This is kind of like the display buffer in graphics rendering. The transmitter sends out the data from the buffer to the receiver. The receiver notices there's activity, then fires up the circuitry to capture data The receiver polls the data line at the character rate. So if the data rate is 1 Mbd, then it will check the data line 1 million times a second. Data is captured into a buffer, then sent off to whoever is supposed to take care of it. In parallel data transmission, for every bit in the buffer, there's a data line going out from it to a receiver's buffer. So if the system was a 8-bit parallel data line, it would have a byte sized buffer and each bit is connected to the other's bit line. At appropriate points in the transmission, the receiver will copy what's in the buffer to somewhere else.   In serial data transmission, each bit in the buffer is rotated out like a queue. The receiver queues up incoming bytes and at appropriate points, copies the data out to somewhere else.   What are the issues with parallel communication then? The biggest one with parallel communication is that it requires one data line for every bit of data being sent at once. 8-bit parallel buses need 8 data lines, 16-bit needs 16 data lines, 32-bit data needs 32 lines, etc. To maintain signal integrity so that the receiver knows how to reference the voltage, typically each data line has its own ground line. You can usually get away with less, but for high-speed or lower voltage buses, you really want one ground per data line. So take what I said previously and double the lines. This is presents a problem not only with the cables, but the connector size as well. SCSI grew up to an 80-pin connector, and well, take a look at it:   By Shieldforyoureyes Dave Fischer - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=4671737   The second is that signals can arrive at different times. Electrical communication may seem instant to us, but in when the time domains are nearing nanoseconds, it starts to matter. The reason for this delay is either due to components holding up the signal to the length of the wire itself. The receiver can't know a data bit is late, so it'll take whatever is in the buffer at the prescribed time.   How does serial communication solve the issues of parallel communication? The first is obvious, serial communication requires at most two wires total: signal and ground (ground is needed to reference the voltage). The second is that since you don't have the requirement of multiple lines needing to have their signals arrive at once, you avoid the issue of late signals altogether. As you can embed a clock signal in the data itself, the receiver need not even know the transmission rate beforehand.   However, when signals are low voltage (typically 5V or less), noise becomes a concern. In cases like this, the data is sent out as a differential pair. That is, there are two data lines, both carrying the opposite signal. If the transmitter sends out a data bit 1, it will send out +/- the voltage of the signal. For example, with USB, a data bit 1 will produce a +5V and -5V signal. The receiver then compares the two signals. If they aren't complementary, the receiver knows that bit is noise and not actual data. While this still carries the problem of signals not arriving at once, 2 lines is much easier to ensure than 8, 16, or 32. Differential signaling is also helpful in that it generates what is called a DC-balanced signal, that is if you took both signals over time and averaged it, the signal would look like a straight line. In this case, 0V.   The bit about differential signaling is important, because not only does it help prevent outside noise from ruining the signal, but at higher baud rates, the data lines themselves generate noise. This is why the Parallel ATA interface went from 40 pins to 80 pins. The extra 40 pins were ground to help shield the signals from themselves.   But aren't some buses like PCI Express, Thunderbolt, DVI, etc. parallel then since they have more than one data line (aside from transmit/receive)? While it's true that, for example, PCI Express x16 has 16 data lines (32 total for transmit and receive), it's not a parallel interface. The way the above interfaces work is they aggregate transmitters. It's like how RAID 0 in drives work. The transmitter can either stripe data chunks and the receiver can sort them out later or each transmitter can handle a single request. This is how PCI Express can have multiple lanes. You cannot, for instance, take a 32-bit parallel interface, chop off half the wires, and expect it work since it was designed to have all 32-bits be sent out and arrive at once.

M.Yurizaki

M.Yurizaki

 

The Adventures of Adding Thermal Pads to an EVGA GTX 1080 SC

tl;dr - things to know about installing the thermal pads Your back plate may have extra padding where you need to install the back plate thermal pad. You'll have to remove these before applying the thermal pad. The back plate has plastic covering parts where the thermal pad will be installed. This plastic is a pain in the rear to remove and leaves a obnoxiously hard to remove adhesive. If you want to feel like the thermal pad will actually do something and you want to remove it, be prepared to spend a lot of time scrubbing off this adhesive. I didn't have any Goo-Gone and alcohol makes the adhesive more gooey, but not necessarily easier to remove. The LED power connector is a pain in the ass to remove. The reason is the connector is barbed so do not attempt to unplug the connector normally. Remove the fan header first, then flip the heat sink over to expose the LED power connector. Use an X-acto knife or something to shave off the barb. EVGA sent VRAM pads for mine. You may get them too. Remember to clean the VRAM chips before applying the new pads. After reassembly, spin the fans. One of the cables may have come loose from its guiding and may hit the fans.   A few days ago I got my thermal pad kit from EVGA for my GTX 1080 SC. Since today's the start of my weekend, I went ahead and installed it. While this shouldn't have taken as long as I took, I spent more time doing something other than installing the thermal pads which I'll go into detail later. Otherwise, you're probably looking at 30 minutes if you know exactly what to expect.   Also I guess this is a good time to show off my computer.   Step 1: Getting to the video card   Opening up the Silverstone FTZ01 leads into a nice compartmentalized motherboard area. I tried to keep the 3.5" bay area free of clutter. I feel the case fan and CPU fan do redundant work since they're pointed in the same direction, but hey, the CPU is always sitting at a nice cool temperature (then again, have a 65W processor helps)   So after unplugging things and taking out six screws holding the left side to the case and nudging it out of the PCIe slot.     I don't like working with the video card if I can help it. Now you can see why.     And there it is in all its glory!   Step 2: Disassembling the cooling system   [Insert picture of backplate here]   The backplate was easy to remove. Something to note though is there's some extra padding that you'll want to remove later. I forgot to take a picture of it, but here's one instead that I edited. The green is where I found extra padding that should be removed later (the rightmost one may not exist, I forget if it was there). the red ones are thermal padding, you should leave that alone.     So after removing the backplate, I get this:     And removing more screws, flipping the card over...     I'd like to point out something. See the open connector closest to the camera? That's the fan header. It takes a bit of work but you can pry it up and off. However, the wires away from the camera? This is a huge freaking hassle to take off. That wire powers the LEDs. Why is it a hassle to take off? EVGA in their infinite wisdom made the connector barbed. I'm not angry at them for the whole thermal pad thing. I'm angry at them for making this connector a "set once and never remove" kind of deal. Yes I'm that mad!   Anyway the way I got it off was I took an X-acto knife and shaved off the barbs until I could get the damn thing off. Please EVGA do not do this again. Actually any AIB, if I catch you doing this I may have to go find your lead engineer and beat them over the head with the card and tell them to dissemble every card that comes in for an RMA that needs cooling disassembly   *Ahem*   Once we have the heat sink off, we have the card in its glor--- Is that a PCIe 8-pin header on the left there? Ya know EVGA, you could've just put the header there and saved us a bunch of trouble with your adapter thing... never mind.     Step 3: Adding the thermal pads   At this point you should rub down the VRAM with alcohol since you need to add new thermal pads to them. I made the mistake (?) of not letting the card cool down completely so they were kind of icky. For cleaning the GPU itself, I used Arcticlean from Arctic Silver. It leaves a nice lemony scent.   Here's the GPU all cleaned up and the VRAM chips with their new thermal pads. Now it's time to put the front plate back on and get the thermal pads for the chokes on.   (also look how clean that heat sink is!) I did one more glory shot before adding some AS5. I added a glob of it which I thought would be best for a GPU.   The last step was to install the larger thermal pad on the back. I don't have pictures of this because here's what happened: Remember the padding I mentioned? I pried those off because I figured the thermal pad would fill in that gap. I realized then that a lot of that backplate has a plastic cover on it. I thought to myself "this thermal pad is obviously interfacing the PCB to the metal backplate and the plastic is just going to get in the way." I started to remove the plastic, which left a bunch of adhesive on it. Apple be jelly of EVGA for their annoyingly hard to remove adhesive. I spent the next half hour or so trying to scrape the adhesive off. I managed to get what I think most of it of, but urgh. I gave up, applied the thermal pad, and called it a day. And this is why I didn't take pictures. I was too annoyed and butthurt about this adhesive.   After putting it back together and getting it in the case, I turned the computer on and heard this rattling noise from hell. That could only mean one thing! The fans somewhere are hitting something! So after taking out the video card I found what was going on: one of the fans was hitting that damn LED power connector: It's a little hard to see, but you can see the left side the blade hitting the wire. There was also some kind of tape or something to keep the power and ground wires together and that's probably what was also being hit. In my effort to manhandle this LED power connector, I must've pulled it from some wire guiding in the shroud. So if you hear a rattling sound from hell, make sure your LED wires are tucked away.   Step 4: Testing the card out   After all was said and done, I put the computer back together, powered it up and...     Holy low temperatures Batman! 31C in about a 25-26C environment! So I let the AS5 bake for a few minutes then fired up 3DMark to run Sky Diver. The results came back... with 5000 fewer points. Well crap! But this was with my case laying flat, I set it up vertically and the score was back to normal. I guess laying the case horizontal chokes the video card enough.   Then for a reasonable stress test I fired up Heaven for half an hour while I let GPUz log the sensor data to check for anomalies.:   So far so good. It was maintaining boost clocks throughout, the memory clock was the same (so those thermal pads were working), and everything else was all green. The PerfCap reason was either due to voltage/power limits or underutilization, with a bit of the card hitting maximum voltage for flavor. So overall I call this a success.

M.Yurizaki

M.Yurizaki

×