Category Archives: Technology

Posted on
by

In my previous article, I speculated on using OpenAI’s LLM API for service selection at the day job. Idea being to guide a prospective researcher into the appropriate services from our catalogue to conduct their research.

I bashed something together on a frustrating Friday afternoon, and it seems to work quite well!

To accomplish this, I pre-load the chat with some instructions, e.g. “behave like an expert structural biologist with experience of all suitable technologies” which is surprisingly straightforward.

The secret sauce comes when it comes to loading the catalogue. Basically, I pull all the services and construct some JSON consisting of the names, the description (which is useful for understanding the limitations of the technology), and a “apply url”. I instruct the LLM to select appropriate technologies from the list, and then give me a valid application like where appropriate. These links will automatically create a new proposal in our access management system, ARIA, so that the user can directly apply.

I ask the LLM to consider alternative technologies that the researcher may not have considered, as well as pipelines where appropriate.

Initial tests are… very positive, I have to say!

This was of course an experiment, and there’s a question as to whether we roll this out on our live site (not least of which the European Commission is … touchy… right now about using US based LLMs), but… not bad for an afternoon’s flight of fancy.

Next, I’m going to possibly start looking at vector databases and how they could be applied to the statistical analysis of deposition metadata, but this might be better directed towards some lucky individual’s PHD thesis.

Posted on
by

What a difference a year makes… and I ain’t dead (yet). Long time, no post. Suffice it to say it has been a very busy year, with a lot going on, both professionally and personally, both good and bad. But, I won’t go into that here.

AI is the new hotness, as has been remarked. A lot of my day job now involves AI in some form or another, and indeed, the field of Structural Biology leads the way on a lot of the technological adoption. Notably, things like AlphaFold for AI based structural predictions, but also, some of the stuff I’m more directly involved with which aims to build a model for improved fragment based drug discovery. More on that stuff in due course.

Compared to that, this little thing I put together while bored during a meeting today may not be earth shattering, but as a proof of concept I think it does start to address a problem we on the ARIA team have faced for a little while.

Namely, our catalogue is very well placed to allow people who know what they’re after to get what they want, but not if they want to, for example, find something out but have no idea how to go about it.

Previously, we had used human experts, and even played with a live chat feature. None of this really scaled very well, but now we’ve got fancy pants AI, I figured I’d give it a try!

My thinking on this is fairly simple, the first step is to make sure all the services in ARIA have good descriptions, including down to what each machine can and can’t do. The better the description, the better the results from the model.

The second is to feed this context into the prompt, and instruct the AI (via the new fangled technique of prompt engineering) to act as an expert and set the parameters according. For example, to instruct to only recommend information in the context, and to provide, as a result, a list of suitable application URLs (which in reality will allow the user to directly apply for the service).

I also instructed the AI to consider how it could connect services together into a pipeline, as well as to consider alternative recommendations.

Proof of concept right now, but already I’m getting possibly interesting results. I’ve got more tedious meetings coming up, as well as a fair amount of sitting in airports, so my plan is to wire this up with a chat interface and let real scientists play. No doubt we’ll have to tweak the prompt a number of times as we go, but as a first play with the technology I think this was a morning well spent!

» Visit the project on Gitlab...

Posted on
by

I have a hetzner server (the server that hosts this blog, as it happens) which runs Debian. While waiting to give a presentation at the day job over zoom, I decided this was a good moment to upgrade the system from Debian Bullseye (now EOL) to Bookworm.

This is something I’ve done before, so I didn’t anticipate this to be a problem.

I was wrong.

Anyway, after performing the upgrade in the usual way, I rebooted and was greeted with… well… nothing.

The server failed to restart.

I logged in to hetzner robot and booted the rescue image, but I couldn’t find anything of note in the server logs. Indeed, the logs appeared to be completely untouched. To me, this pointed to a problem with the boot loader.

Thankfully, Hetzner supports a vKVM, so I booted into that. Again, nothing.

Hmm…

On playing around, however, I did notice that if you boot the KVM (which automatically starts the rescue image), and then trigger a soft reset from within the kvm itself, the kvm will remain attached during the boot process, and allowed me to see… a grub error.

Joy. But at least I had identified the fault.

The error in question was complaining about not being able to find normal.mod, which is fairly critical. I poked around the recovery console, mounting the various filesystems in the RAID, but couldn’t find the file. So I attempted to load the kernel manually using insmod… to be treated to another error complaining about linux.mod not being found.

So, grub was completely b0rked.

This, however, gave us the answer….

The fix

The problem is that, for whatever reason, grub (the bootloader) has got messed up. So, we need rebuild it. Since we can’t boot the system, we need to do this from the rescue system. Hetzner does have a installimage tool, but I felt a little wary about running that since my understanding was that this would wipe everything… a bit of a nuclear option.

Thankfully, there was a lower impact solution we could try first.

  1. Confirm your software raid is working by taking a look at /proc/mdstat, and listing the structure using lsblk. Your raid should already be assembled, but if it isn’t, you can run mdadm --assemble --scan
  2. Next, find your mount points, for me:
    • md0 = swap (ignore)
    • md1 = /boot
    • md2 = /
    • md3 = is your home directory, so leave this alone
  3. Now you’re ready to rebuild your filesystem in chroot.
    • Mount your root drive (md2) to /mnt/ – mount /dev/md2 /mnt
    • Mount your boot drive (md1) inside – mount /dev/md1 /mnt/boot
    • Bind various system drives
      • mount --bind /dev /mnt/dev
      • mount --bind /proc /mnt/proc
      • mount --bind /sys /mnt/sys
    • Finally, create your chroot: chroot /mnt
  4. Now, rebuild and reinstall grub
    • grub-install /dev/sda (I took a guess this was where my boot loader is, usually the case)
    • update-grub
  5. Exit, unmount, and reboot
    • umount /mnt/dev
    • umount /mnt/proc
    • umount /mnt/sys
    • umount /mnt/boot  
    • umount /mnt
    • reboot

All being well, your server should be back up and running. For me, however, this wasn’t quite the end of the story.

After rebooting, my server was still inaccessible. I repeated the vKVM trick and fully expected to see a grub error, however the server was booting normally.

Using the root password, I logged in to the console and sure enough my server was running, however there was no network connectivity.

A bit of poking around shows that for some reason the network interface name had changed, and the server was hard coded in /etc/network/interfaces to use the incorrect one.

I used ip link show to find the correct network interface address, modified interfaces and restarted.

Boom, server back up… and now I can tell you about it here!

Hope this is of use to someone.