Prind Broken After Unexpected Shutdown

I use prind to run my 3d printer. I’ve had this happen twice now where I get this error after an unexpected shutdown (like a power outage):

“Moonraker can’t connect to Klipper! Please check if the Klipper service is running and klippy_uds_address is correctly configured in the moonraker.conf”

I looked at moonraker.conf to check the address, and then I exec’d into the klipper and moonraker containers to make sure they could both see the klipper socket. I did a “touch” on a file from one container and could see it in the other container.

I finally started digging around in the docker volumes and figured out I had two volumes in an odd state:

prind3_log

prind3_run (this is the one that contains the klipper socket)

Sidebar here: I had this problem a few months ago and fixed it by reinstalling prind. I ended up reinstalling it in a folder called “prind3” and I was completely flummoxed that the new installation stopped working if I renamed it back to the original “prind” folder name. As you can see, the volume name format is <prind folder>_<container name>. So when I renamed the working installation back to “prind”, it stopped working because it went back to using these broken volumes.

I was not able to delete the volumes using docker volume rm <volume name>–I got an error about it being used by an active container (LIES!)

I was able to delete the volumes from the file system by removing the folders (as root) under:

/var/lib/docker/volumes

However they still showed up when I ran docker ls. I ended up having stop all the container with docker-compose, and then restart the docker service. At this point the phantom volumes were gone, and when I started up Prind everything was working.

This stackoverflow post helped steer me in the right direction for how to remove the phantom volumes. The way I found the phantom volumes was by running:

docker volume rm $(docker volume ls -q --filter dangling=true)

followed by:

docker volume ls -q --filter dangling=true

Which showed a couple remaining volumes, which as I mentioned earlier I wasn’t able to remove without extraordinary measures. There was one casualty here–since prind was not running it clobbered the volume that stored all the gcode files I had uploaded to Moonraker. Not really a loss in my case.

Hiring a Surveyor

I recently hired a surveyor to figure out exactly where my property lines were.  I’m in Seattle, Washington in the USA–this is about my experiences here.  Your experiences may be a little different in other states, and a lot different in other countries.

The hiring process involved a couple of false starts and some confusion.  I finally found a good surveyor, but I learned a lot about how the process works and how one might shop around for a good survey at a good price.

When I was first getting quotes for a survey I was shocked at the price difference–it was between about $1500 and $5000 for the same quoted work.  That’s a big difference!

Between some of my own poking around in the county property records and chatting with my surveyor I figured out that you want to hire a surveyor that’s worked near your property before.  I’ll explain a little more about why this is and then explain how you can figure out which surveyors have worked in your neighborhood before.

How Surveyors Work

The government (federal, state, etc.) maintains a number of survey monuments–these are documented, fixed points from which a surveyor calculates distances and angles to figure out your property lines (there’s lots of information about this elsewhere).  But here’s the key–surveyors don’t work directly off the monuments if they don’t have to.  Monuments tend to be in inconvenient places like the middle of the road.

So when a surveyor comes to a neighborhood for the first time they will usually use the monuments to establish their own “control” points.  Then they use these to do the actual survey.  If they come back to the area later for another survey, they can reuse these control points instead of having to work from the monuments in the middle of the road.  My surveyor (in 2017) said he used some control points that had put down several years before when he was doing a survey in the neighborhood.

The cost of a survey is generally related to how much work the surveyor has to do to find good control points.

How to See Surveys from your Neighborhood

This is the hard and frustrating part.

Surveys are often (usually?) recorded with whoever handles property records in your area (“recording” is a process of filing an official document with the government to make it a public record).  Here in Washington state property records are handled by the Recording Office of the county the property is located in.

Basically you want to search for recorded surveys near your property–survey maps always show the date and name of the surveyor.  You want to find a surveyor that’s worked near your property recently.

Seattle is in King County, and our Recorder’s Office has a web search. I think most counties do these days.

To start, you need to find the “legal description” of your property.  You can generally find this on your deed.

From here there’s a lot of trial and error involved in finding records near you.  Your legal description will generally show subdivision (usually some sort of fancy name like “lake front addition”) and block–you can search on this but it’s a pretty small area.  When I use the King County records tool I select “advanced legal” search type to be able to search on these fields (and the ones I mention later here).  You also want to select “map” for document category.

King County has a “parcel viewer” that lets you get some extra information that can help in a search (http://gismaps.kingcounty.gov/parcelviewer2/).  You can plug your address in here, drill down to property details, and get the quarter-section-township-range of your property.  This gives a much larger area to search for.  Note that leading zeroes matter for this kind of search (i.e. the parcel viewer might say range “4” but you want to search on “04”.

The quarter-section-township-range search  is good (if it doesn’t produce too many results) but if you’re on the corner of one of these areas you’ll miss nearby surveys.  You may have to dig into plat maps to figure out the search terms.  In king county you can get a plat map of your area by plugging your parcel number (available from your property tax bill or the parcel viewer above) into this site:

http://info.kingcounty.gov/assessor/emap/default.aspx

Side note:  I noticed the quarter search in king county records is odd–a quarter is actually pretty big so the search allows you to search by quarter of a quarter.  e.g. if you’re in the NE corner of NE-24-23-03, you would search for NE-NE as the quarter.  My property was NE-NE, but I was on the south boundary, so I had to search for NE-SE to find a survey for my neighbor’s property to the south.
I also found if I just searched for quarter NE I got one result–changing the search to NE-NE produced many more.

So…this part will be the most time consuming and frustrating, but eventually you should end up with a few surveys done on your block, hopefully within the least ten years or so.  The survey will generally show the name and contact info of the entity that did it.

When you contact the surveyor, it might not hurt to mention that they’ve worked your neighborhood before, say something like, “I also see from public records that your firm did a survey for the property two houses down from me in 2008”.  This might get your quote moved to the top of the pile since they know it will be easier to look in their own records and come up with a quote.

 

Bricked Wink Hub–Solid Yellow and 3.9.4 update

(I’ve also posted this to the rootwink.com forums as mattjm, but post approval seems slow these days and I want to get this information out there)

My hub bricked on the last update. It was rooted at one point, but I started updating via the app and gave up on root somewhere around version 3.4 But…I tried to update a couple months ago and it bricked. The symptoms were it would boot normally, then the LED would go solid yellow (not a documented state). I was able to regain access via serial and restore it to version 2.68 using a cf_url and cf_cert file I found online. But updates via the app continued to result in the bricked, solid yellow state. Looking at the logs from updating via the app, it seems the error was “failed to format flash” or something like that.

I found out that when I trigger updates via the app, it updates cf_url and cf_cert to what I need for the latest version. I was able to get the contents of these files and update via the command line. I’ve outlined the steps below, in case someone else has experienced this. This procedure doesn’t maintain root–you may or may not be able to maintain it by following some of the other instructions on rootwink.com. This will work from version 2.68 and get you to 3.9.4. It will probably work for other versions, too. (Test from 2.68 because that’s what I was able to find a cf_url and cf_cert for to get it back to a working state)

This was the procedure to fix:

1. re-establish root through UART pins. Set a root password or authorized_keys file.

2. Reboot normally and log in as root.

3. command line:

sed -i 's/reboot/#reboot/g' /root/platform/*.sh

(command borrowed from rootwink.com post). This just stops the scripts from rebooting automatically after they run.

4. Put the following contents in /database/cf_url:

sw_pkg_url=https://hub-updates.winkapp.com/wink-hub/3.9.4-0-gc59efe4ee8-hub-app-7c6bb7d1-d7d5-114a-c368-5c4d14a3185e/upgrade_manifest.txt
md5_manifest=3311d5788940a4ad76fd0e585533d7b0

5. Put the following contents in /database/cf_cert:

-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAumZE0qsNgtTQNCOhqx8+
+3DQG33HIvgqdPJ9qa4A9rdtuTADJ3FUnO53+EQYFuZaZ8RK76DQddtqCUzRyra7
JI9RTXxYX4VWxTMEfKMoi2R79BFPOXg61Q3ZCZ4RP2gz1Mkso59ULNziLmeSQwSK
udeTSmkcLUql9Ygtx8iro+JiFBr6wwyLFm+KblkRvk7aYjctdJvGOGUfoYKyKQJz
5mXQiHoCqpkqdNL436paQjsxIdcEqu+iRl8n2BIgDqxHg1ZTdiNe6NIyohkaG97r
Rfzv2LZQ9zgfdQfGArwc9B+AGyKFJ4FgW0SdCmZSNrpqoEVCS1Xuw8QLe6oraMuC
MwIDAQAB
-----END PUBLIC KEY-----

6. Note cf_url and cf_cert don’t generally exist unless you’ve triggered an update through the app. If you want a newer version, you can log in as root and trigger the update through the app (first run the sed command above to prevent reboot)–you can cat cf_url and cf_cert to get the contents, then run this procedure again.

7. command line:

/root/platform/upgrade.sh

You should see the standard update stuff scrolling by

8. Manually reboot

9. On reboot some additional updating should happen. Then you should get the solid blue light with version 3.9.4

Windows Certificate Store Permissions

The Windows certificate store, especially private key permissions, has caused no end of misery for people the world over. I recently discovered something that seems to get around private key permission nonsense. I’m not sure why, but it works. Note this only works with .NET Framework 4.6 or later.

Suppose you already have an X509Certificate object called “cert”. Running this code once you have the object seems to magically make permission issues on the private key disappear (note you need “using System.Security.Cryptography;”):


X509Certificate2 samecert = (X509Certificate2)cert;
RSA rsaobj = samecert.GetRSAPrivateKey();
((IDisposable)rsaobj).Dispose();

If you’ve dug enough into the code or documentation enough to understand what’s going on here, please comment. I have a few ideas but nothing I could write coherently about right now. I sort of suspect this was always the way it was supposed to work…

HIPAA Rule Change — guns and mental health

I decided to dig through the HIPAA rule change issued by Health and Human Services 2016-01-06. I’ve read a lot of concerns about how your shrink will report you to the FBI, and lots of other crazy ideas. Here are some facts.

This rule doesn’t change who is prohibited from purchasing a firearm. All it does is provide a mechanism whereby health providers who possess knowledge about someone who is prohibited from buying a firearm under existing law may report that information to the FBI’s background check system (NICS). That’s it. Your doctor or shrink cannot report you to NICS for taking antidepressants or other drugs, or even talking about suicide. A “lawful authority” must make a determination that you are mentally incompetent before the information can be reported to NICS. This is purely a bookkeeping rule to help make sure that existing information is reported to NICS.

The rule is published on the HHS Website. Here is the most important bit:

This final rule applies only to covered entities that function as repositories of information relevant to the Federal mental health prohibitor on behalf of a State or that are responsible for ordering the involuntary commitments or other adjudications that make an individual subject to the Federal mental health prohibitor.

And if you’re wondering what the “Federal mental health prohibitor” is:

Among the persons subject to the Federal mental health prohibitor established under the Gun Control Act of 1968 and implementing regulations issued by the Department of Justice (DOJ) are individuals who have been involuntarily committed to a mental institution; found incompetent to stand trial or not guilty by reason of insanity; or otherwise have been determined by a court, board, commission, or other lawful authority to be a danger to themselves or others or to lack the mental capacity to contract or manage their own affairs, as a result of marked subnormal intelligence or mental illness, incompetency, condition, or disease.

The rule linked to above includes a lot of discussion about the current state of reporting and how the current rule was arrived at. It’s enlightening but dry.

Ford EEC-IV computer on bank-fired EFI trucks

I spent 3 + hours trying to get my truck to run a cylinder balance test…I finally decided that it just wasn’t going to do it, and on further research I discovered the cylinder balance test is only available for Sequentially Fuel Injected (SEFI) engines. The older Ford trucks are all bank-fired. Bank-fired means the injectors fire in sets of four (on the V8) instead of each cylinders injector firing at the appropriate time.

Also, the order of operations for reading codes on my truck seems to be a little different than most of the tutorials I found. I have a 1987 F-150 (302/5.0L), and this is the order of operations for the Key On Engine Running (KOER) test:

1. Set up the voltmeter (analog works best–I wasn’t getting good reads with my digital voltmeter). there are lots of other sites that tell you exactly which pins to short and connect to for the test.

2. Start the engine. After a few seconds you’ll get the engine ID pulse–it should be the number of cylinders divided by two (e.g. four pulses for a V8)

3. After the engine ID pulse you have ten seconds (or so a lot of sites say….I did these things in about five seconds) to do the following:

-Depress the brake pedal
-turn the steering wheel 1/4 turn left and right (some sites say 1/2 turn one direction….I’m not sure it matters as long as you turn it enough to give the sensor a pressure reading)
-press and release the overdrive switch if you have one (I don’t)
-A lot of sites say to do the dynamic response test (briefly press the throttle to WOT) here. I didn’t find to be the case. My ECU asks for it later.

4. About 70 seconds after you start the engine, you’ll see a single pulse. This is the ECU asking for you to briefly go WOT for the dynamic response test (just push the pedal all the way down and let go).

5. Shortly after this (within 10 seconds) you should start getting codes in the usual fashion. You’ll see a code 77 if you didn’t do the dynamic response throttle press when it was requested.

6. On bank-fired EFI trucks, no amount of fooling around at this point will get you a cylinder balance test. It’s just not available on this system. I think this is because the test works by shutting off fuel to a specific cylinder, and that’s not possible on a bank-fired system.

Hope this helps someone….

Motorcycle chain alignment

I seem to have a hard time with this, and I’m too cheap to buy this tool:

http://www.motionpro.com/motorcycle/partno/08-0048

(I’m also not entirely sure how effective it would be…I’ve heard mixed reviews). Anyway…the best thing I’ve found so far is from this post by user Syscrush on SVRider:

http://forum.svrider.com/showpost.php?p=1147779&postcount=12

What I like to do is tighten the left adjuster first until the chain tension is right, then put a straightedge on the sprocket and tighten the right adjuster until the face of the straightedge that’s against the sprocket lines up with the inside edge of the inner sideplates on the chain.

Seemed to work well for me….better than other methods I’ve tried, anyway.

Online Backup Solutions

I have been evaluating online backup solutions. Currently I backup to an external hard drive. This has two problems:

1. I backup manually and via a script. I don’t do it very often.
2. All backups are onsite. It’s been years since I burned a DVD and dropped it at my parents’ house.

So, I finally decided that I needed automated and offsite backups. After a week and half of research and testing I decided to purchase Mozy.

Read on if you want a more comprehensive review. There are many offerings out there right now. Wikipedia has a list:

http://en.wikipedia.org/wiki/List_of_online_backup_services

I only looked into a handful of these, and looked especially closely at the two major players–Carbonite and Mozy. I should also note I’m running 64-bit Windows 7.

I’ll go over Carbonite and Mozy first, then mention the other services I looked at. Keep in mind that my requirements may not be the same as other users so your mileage may vary. I was looking for a very simple solution–simple because I am lazy and won’t do backups if it takes any effort, and simple because I wanted a product that I could recommend to less technical friends and family.

Carbonite generally had slightly better reviews than Mozy, and seemed the front runner for a while (I had the trial installed for about a week). However…in the course of my research I discovered that it excludes certain file extensions from all backups unless you tell it to include them. Including them is a manual process (there is no “hey please backup everything I tell you” button). To their credit, they do mention this on their website (mostly), and will send you an exhaustive list via email if you ask (I asked and they sent it to me promptly). Here is the help article where they talk about this:

http://cp-carbonite.kb.net/article.aspx?article=1069&p=3

To be fair, the automated backup sets in mozy only back up certain files, similar to what carbonite does. However, if you tell mozy “back up this folder”, it will back up everything in that folder. If you tell Carbonite to backup a specific folder, it will happily exclude a bunch of files by default. Video files are not backed up by default in Carbonite, either, which seems pretty bad to me–how many people have digital camera movies on their computers, or videos from their phones? Here is the default Mozy policy for automated backup sets:

http://support.mozy.com/docs/en-user-home-win/faq/concepts/backup_sets_default_contents.html

Anyway, these exclusions were pretty much a deal killer for Carbonite. Remember how I said I want a simple solution because I’m lazy? That extends to setting it up–if I say “back up this folder”, I don’t want to have to worry about whether or not certain files in that folder are getting backed up. Same thing if I am recommending a backup product to friends or family–I just want them to be able to check a box to make sure the videos of their kids first steps are backed up.

As far as retention of deleted files, Carbonite and Mozy both have deleted files available for 30 days after you remove them from your computer.

Backup with Mozy was much faster than with Carbonite. However, I was on a free trial with Carbonite so that could have affected the speed. And honestly, I’m not terribly concerned with the speed at which the initial backup occurs (I’m backing up about 40GB).

The test restores I did with Mozy (~1.7gb) during the free trial had some issues–about 400 files didn’t make it back. Eventually I got them via a “web restore”. This is a clumsy method but it did get all the files back–and that’s what’s really important in a disaster recovery scenario. I emailed Mozy support about the restore issues and they seemed fairly responsive but also seemed to give mostly “reading from the binder” responses. This was expected considering who their target audience is. UPDATE: I did another restore with mozy on a different computer, this time 2.8GB but a smaller number of files, and didn’t have any problems restoring via the Mozy client.

So there’s the head-to-head. Honestly, if it weren’t for the default file exclusions, it would probably be a toss-up between carbonite and Mozy.

Other services I looked at:

I considered crashplan. They offer unlimited backup at a similar price point to Mozy and Carbonite, but the “free” client is ad supported and it costs $60 for the “pro” version. This was enough that I didn’t even try them.

I installed idrive on one computer. The interface was hideous and complex and I promptly uninstalled it. I would not recommend this to my non-technical friends.

I briefly played around with Jungledisk. You pay per GB for their client to store data with Amazon S3–If you store 20GB or less it is similar in price to Carbonite and Mozy. It seems like a decent system–I decided not to go with them mostly for cost reasons.

SOS online backup had great magazine reviews but awful user reviews. I steered clear. It was also more expensive than other services. One feature SOS touted that other services do not have is unlimited versioning of files (within the limits of your storage space, of course).

Iron Mountain has an online backup service. This is the company we use for off-site tape storage at work. They seem very professional and I expect their online backup service is excellent. I can’t afford it, though; $9/month for 2 GB of backup. If I had mission critical data I would look at Iron Mountain first.

Backup Integrity: From what I’ve been able to ascertain, the online services use RAIDed trays of disks and no off-site backups (I suspect this would not be the case for Iron Mountain, but I haven’t contacted them). If a disaster (earthquake, fire, etc) wiped out the datacenter, and then your local drive crashed, you’d be out of luck. Statistically, the odds of this are pretty low, but it’s something to keep in mind. A backup to an external drive every six months or so isn’t a bad idea, even if you’re using one of the online backup services.

Feel free to post any comments on your own experiences.

Client Certs and Windows Server 2008

I just spent a while tracking down a cert problem while migrating an app to IIS7 on Windows Server 2008.

This app taps into a web service using a client cert. I finally tracked the problem down to a permissions issue with the private key on the client certificate. On Server 2003 and earlier these permissions are managed with the winhttpcertcfg.exe tool, but that’s not available on Server 2008 (or at least not supported, as far as I can tell). Turns out it’s actually pretty simple, though. See screenshot below:

Windows Server 2008 Client Cert

Yeah…just a simple right-click on the cert in the certificates snap-in. Also, for getting SSL traces, this blog had a good system.diagnostics section:

http://blogs.msdn.com/asiatech/archive/2009/04/08/using-system-net-trace-to-troubleshooting-ssl-problem-in-net-2-0-application.aspx

The microsoft documentation, as usual, was lacking, and most of the examples I tried didn’t seem to actually output the trace to a file (at least not where I expected to find it).

keywords: server 2008 client cert certificate permissions winhttpcfg

Diving

I bought a drysuit earlier this week, and went diving with it today. These pics are from suiting up for the drysuit orientation Matt gave me. Diving a drysuit takes a little bit of extra training, mostly in what to do if you start getting air in your feet (because you can end up rocketing to the surface if you don’t take of that problem real quick). I have to say it was slightly amusing popping out of the water fins first looking like the Michelin man.