FAQ Overview

Software

How do I show disk usage breakdown including hidden directories?

Using "du" with the following options will show usage of all files and directories including hidden (those starting with a dot).

du -sch .[!.]* *

Author: Chris Burton
Last update: 2018-08-30


How do I disable automount in Raspberry Pi OS (Bullseye with GUI)?

Create a file ~/.config/pcmanfm/LXDE-pi/pcmanfm.conf for the "default" user (the one created when installed) with the following contents.

[volume]
mount_on_startup=0
mount_removable=0
autorun=1

After a reboot USB disks/etc. will not be automatically mounted anymore.

Author: Chris Burton
Last update: 2023-07-16


Hardware

How to I setup a login prompt using the USB-USB Null Modem?

To allow you to login to a Linux machine over the USB-USB Null Modem a getty needs to be ran on the end you're trying to log into. In this guide I will be logging in from the B end "From" to the A end "Target".

These instructions should work on both CentOS 7 and Ubuntu 18.04.3 LTS

On both sides tail /var/log/messages (CentOS) or /var/log/kern.log (Ubuntu) and plugin the adaptor watching for the new devices entries which will look something like this.

kernel: usb 1-12: new full-speed USB device number 9 using xhci_hcd
kernel: usb 1-12: New USB device found, idVendor=3171, idProduct=0032, bcdDevice= 1.10
kernel: usb 1-12: New USB device strings: Mfr=1, Product=2, SerialNumber=3
kernel: usb 1-12: Product: USB-USB Null Modem A
kernel: usb 1-12: Manufacturer: 8086 Consultancy
kernel: usb 1-12: SerialNumber: 0123
kernel: cdc_acm 1-12:1.0: ttyACM0: USB ACM device

From the above log lines take note of what your machine uses in place of "1-12" and replace that in the examples below (for each side).

On the target machine create a new udev rule to rename the interface. Edit /etc/udev/rules.d/serial.rules and add a line similar to this.

# Serial Console Login:Target / Access:From
SUBSYSTEM=="tty", ATTRS{idVendor}=="3171", ATTRS{idProduct}=="0032", ATTRS{serial}=="0123", SYMLINK+="ttyFromTarget"

Replacing "Target" with the name of the machine you're logging into and From with the name of the machine you're logging in from.  Also replace the serial number "0123" above with the serial number of your device.

Now on the from machine create a new udev rule to rename the interface. Edit /etc/udev/rules.d/serial.rules and add a line similar to this.

# Serial Console Login:From / Access:Target
SUBSYSTEM=="tty", ATTRS{idVendor}=="3171", ATTRS{idProduct}=="0033", ATTRS{serial}=="5123", SYMLINK+="ttyFromTarget"

You will notice here both the serial number and product ID are different to what we added on the Target machine to allow ease of identifying each side of the USB-USB Null Modem.

Reload the udev rules on both machines.

udevadm control -R

If you're running ModemManager this can be stopped and disabled unless you also use Modems (CentOS).

service ModemManager stop
systemctl disable ModemManager

Unplug and replug the USB-USB Null Modem or run the following commands replacing the "1-12" with what you had in the log above (this will be different on both sides).

echo 0 > /sys/bus/usb/devices/1-12/authorized
echo 1 > /sys/bus/usb/devices/1-12/authorized

Now we can enable and star the getty (login prompt) on the Target machine.

systemctl enable serial-getty@ttyFromTarget
systemctl start serial-getty@ttyFromTarget

On the "From" machine you should now be able to connect to the serial device "screen /dev/ttyFromTarget 115200", press enter a couple of times and you should see a login prompt after a few moments.

Author: Chris Burton
Last update: 2019-11-26


Internet

How do I find all sites hosted on the same IP address?

To find all sites hosted on the same IP address is not an easy task due to the way a server can be configured to answer for any domain name even those that do not exist or doesn’t point to the IP in question.

We advise using our DNSHistory tool to find possible sites hosted on the same IP address.

  1. Open up DNSHistory in a new window.
  2. Type in the domain name in question into the search box.
  3. If the search returns a CNAME then click on this until you get to an entry with an “A” record.
  4. If multiple A records are shown you will need to repeat the instructions below for each IP address listed.
  5. Click on the IP address shown under the A record heading.
  6. Next to the heading “What links here by” click on “A”.
  7. You should now be shown a list of all other domain names that are potentially hosted on this IP address.

The list of domains shown is most likely not complete but should give you a good idea of other sites hosted on the IP.

Author: Chris Burton
Last update: 2014-12-16


Business

How do I verify an EU VAT number?

EU VAT numbers can be verified at the VIES site.

Author: Chris Burton
Last update: 2014-12-16


Hardware » VoIP

How do I factory reset a Siemens Gigaset A580IP / C475IP / S685IP

To remove all settings from the phone (It will revert back to DHCP and a default pin of 0000).

 

  1. Remove the cables from the base station.
  2. Put the power cable back into the base whilst holding the blue button for 10 seconds.
  3. Once you release the button the base should be reset.
  4. Check which IP has been allocated by your DHCP server and configure in your browser.

Author: Chris Burton
Last update: 2014-12-16


How do I upgrade the firmware on a Cisco/Sipura SPA942/SPA962/etc using TFTP?

You will need a TFP server to update the firmware.

  1. Obtain the latest firmware for your device from Cisco Support.
  2. Extract the archive and copy the *.bin file to spa.bin and place it in your TFTP servers root directory.
  3. In a web browser visit "http://<PHONE IP ADDRESS>/upgrade?tftp://<TFTP SERVER IP>/spa.bin
  4. Do not power cycle/reboot the phone it will automatically reboot once the firmware has been updated.

Please note the filename for the firmware must be named spa.bin otherwise the upgrade will fail.

Author: Chris Burton
Last update: 2014-12-16


Hardware » Gadgets

How do I pair a Current Cost transmitter with an Envi/CC128 Monitor

Follow this procedure to pair the Transmitter (standard or mini) on an appliance channel.

  1. Press the “Up” or “Down” buttons until you find the Appliance channel you wish to pair the transmitter on. The Appliance channel number is shown in the lower right hand corner of the display (where the main channel shows the current temperature).
  2. Press the recessed button on the transmitter.
  3. Press and hold the “OK” button on the monitor until the red light flashes (within 60 seconds of pressing the transmitter button).
  4. The monitors display will now search for the device and then update the display to show the current usage.

Author: Chris Burton
Last update: 2014-12-16


How do I set the time on the Current Cost Envi?

  1. Press and hold the "OK" button until the red light below flashes.
  2. Using the up and down arrows set the hours (using the 24 hour clock).
  3. Press "OK".
  4. Using the up and down arrows set the minutes.
  5. Press "OK"
  6. The time has now been set on the CurrentCost Envi.

Author: Chris Burton
Last update: 2014-12-16


Hardware » Mobile

Android phone on Three - data has stopped working with 3G.

On the Three network you must use 3G for data access, sometimes the APN profiles can cause problems and they need to be "refreshed" before the phone will correctly work on the Three network for 3G. The following information applies only to the San Francisco phone being used on the UK Three network.

1) Make a note of your existing APN settings (to restore if you experience any problems)

From the main Menu select "Settings", "Wireless & networks", "Mobile Networks" and then "Access Point Names".

For each of the APNs listed make a note of the details and then enter the following details (these details are for the UK Three network ONLY other networks will require different information which your provider should be able to supply). Once you have backed up your existing APNs remove them via the "Menu" button whilst reviewing the APN.

Modify the new APN details as follows (other options do not need to be changed)

Name: Three Network
APN: three.co.uk
APN type: Default

Then create a second APN (for MMS)

Name: Three Network MMS
APN: three.co.uk
MMSC: http://mms.um.three.co.uk:10021/mmsc
MMS PROXY: mms.three.co.uk
MMS PORT: 8799
APN type: mms

With the settings above your phone should reconnect and use 3G for data.

Author: Chris Burton
Last update: 2014-12-16


Hardware » Computers

How do I boot a Supermicro motherboard using PXE?

To boot with PXE you must first enable the network cards BIOS within your system BIOS and then when booting press F11 which will show the boot menu.

Author: Chris Burton
Last update: 2014-12-16


How do I stop X from starting on boot on the Iteaduino Plus?

From the console you can disable X at startup by issuing the following command.

sudo mv /etc/init/lightdm.conf /etc/init/lightdm.conf.disabled

And then reboot the board.

You will then find you no longer have the X graphical system running and an extra ~100MB of memory to use.

If you wish to re-enable X at startup run the following to "undo" it.

sudo mv /etc/init/lightdm.conf.disabled /etc/init/lightdm.conf

And again following a reboot the service will be re-enabled.

Author: Chris Burton
Last update: 2014-12-16


How do I set/reset the Iteaduino Plus root users password?

When logged in as "linaro" you can reset the root password with the following command

 

sudo password

You will then be able to enter a new root password, you will then be able to login directly via SSH/etc. as the root user.

 

Author: Chris Burton
Last update: 2014-12-16


Software » Hosting Control Panels

How do I disable Greylisting in SmarterMail?

To disable greylisting (graylisting) you will need to login as an admin and follow these simple steps.

 

  1. Expand “Domains” and click on “All Domains” and find the domain you wish to disable greylisting for.
  2. Right click on the domain and select “Edit”
  3. Select the “Technical” tab
  4. Check the box for “Disable Greylisting”
  5. Click “Save”

Greylisting should now be disabled for this domain.

Author: Chris Burton
Last update: 2014-12-16


Where can I find the MySQL root user password for DirectAdmin?

How do I get mysqls root user password for DirectAdmin?

The username and password used to access MySQL as the “root” user can be obtained from the following file.

/usr/local/directadmin/conf/mysql.conf

In this file you will see the current “root” username and password used by DirectAdmin to access the database.

Author: Chris Burton
Last update: 2014-12-16


How do I recalculate Plesk statistics for a site?

If for some reason Plesk does not update the statistics properly for a site or you wish to re-run them after changing log files etc it is a simple process.

 

  1. Login to your Plesk server as user “root” using SSH.
  2. Run “/usr/local/psa/admin/sbin/statistics --calculate-one --domain-name=<domainname>” – replacing <domainname> with the domain name you need to reprocess the statistics for.

Author: Chris Burton
Last update: 2014-12-16


How do I fix Unable to connect to database: saved admin password is incorrect (Plesk)?

ERROR: PleskFatalException
Unable to connect to database: saved admin password is incorrect.

0: /usr/local/psa/admin/plib/common_func.php3:190
psaerror(string ‘Unable to connect to database: saved admin password is
incorrect.’)
1: /usr/local/psa/admin/auto_prepend/auth.php3:93

The following instructions assume your Plesk admin user is “admin” and you with to reset the password to “setup”. You can check the password Plesk is expecting to use by doing “cat /etc/psa/.psa.shadow”.

  1. Using your favoured editor create the file “/root/mysqlreset.sql” with the content “UPDATE mysql.user SET Password=PASSWORD(’setup’) WHERE User=’admin’;FLUSH PRIVILEGES;”.
  2. Stop the mysql server “/etc/init.d/mysqld stop” (this will cause issues with any sites utilising MySQL).
  3. Start the MySQL server using the initial SQL file created above “mysqld_safe –init-file=/root/mysqlreset.sql &”.
  4. This should have reset the password and you can now restart MySQL.

You should now be able to login to the Plesk control panel via http://<sitename>:8443/.

Author: Chris Burton
Last update: 2014-12-16


How do I resolve the Error: "please wait for graphics to be generated" on HSphere/SiteStudio?

This normally occurs when imaker has stopped running, you can check its current status by using

 

/hsphere/shared/SiteStudio/imaker.sh status

 

If this shows the process is not running then it can be restarted by using

 

/hsphere/shared/SiteStudio/imaker.sh restart

 

You should now be able to use SiteStudio and images should be created correctly.

Author: Chris Burton
Last update: 2014-12-16


How do I install Softaculous on cPanel?

To install Sofaculous follow these simple instructions:

  1. Login via SSH to the server (as root, or "su - root" if accessing via another user.
  2. Execute the following commands.
    cd /usr/local/cpanel/whostmgr/docroot/cgi
    wget -N http://www.softaculous.com/ins/addon_softaculous.php
    chmod 755 addon_softaculous.php
     
  3. Login to WHM.
  4. Under "Server Configuration" click on "Tweak Settings".
  5. Under the "PHP" heading enable "ioncube" for "Loader to use for internal cPanel PHP".
  6. Click "Save".
  7. Refresh the menu in WHM (normally by pressing F5).
  8. In the "Plugins" menu select "Softaculous - Instant Installs".
  9. Softaculous will now download packages - wait for this to complete (scroll within the frame to check).
  10. Once completed click on "Software" to view the installed software.
  11. If you using the Free licence you can now use Softaculous otherwise continue with the next step.
  12. To upgrade the Licence to Premium click on "Home" under the Softaculous banner.
  13. Click on "Buy Premium License" - you will now be taken to the Sofaculous website.
  14. Make a note of the "Softaculous License" number.
  15. Click on "Buy a license".
  16. Login to Softaculous to proceed with the order.
  17. Click on "My Licenses" and open in a new window.
  18. Enter the license number noted above and "Add License".
  19. Close the window and return to the main Softaculous site.
  20. Enter in the License number noted above and click "Purchase Softaculous".
  21. Select your payment method and make the payment.
  22. In the "Plugins" menu of WHM select "Softaculous - Instant Installs".
  23. Click "Refresh License" which should then show the Premium Licence.
  24. Click the "Software" heading.
  25. Check the box just below "Installed" (this will check all boxes below) and click "Update Settings".

Once the above steps have been completed you will have a fully installed and updated Premium installation of Softaculous.

Author: Chris Burton
Last update: 2014-12-16


How do I update the Plesk limits when the user is over disk quota?

Normally the Plesk control panel will not allow you to update the limits of a hosting account if the account is over quota. This can quite easily be achieved by altering the values stored in the database whilst updating the account.

 

Login to your Plesk server and connect to the MySQL server (psa database).

 

mysql -u admin -p`cat /etc/psa/.psa.shadow` psa

 

Once connected check (and keep) the current disk usage value (replace <DOMAINNAME>).

 

SELECT real_size FROM domains WHERE name='<DOMAINNAME>';

 

You will now see something similar to this - record the value given

 

+-----------+
| real_size |
+-----------+
| 434324252 |
+-----------+

 

Update the value to a lower amount which will allow you to update the limits within Plesk.

 

UPDATE domains SET real_size=0 WHERE name='<DOMAINNAME>';

 

Login to your Plesk control panel and update the account limits.

Once the limits have successfully been updated in Plesk you can restore the old disk usage value (replacing the number for the real_size with that shown in the SELECT query and again replacing <DOMAINNAME>).

 

UPDATE domains SET real_size=434324252  WHERE name='<DOMAINNAME>';

This completes the process and the Plesk control panel should now show the new Limits with the "over quota" disk usage shown correctly.

 

 

Author: Chris Burton
Last update: 2014-12-16


Why is cPanel is showing disk usage when the account has no database?

Sometimes cPanel does not remove the database correctly and leaves behind files.

 

Take a backup and then remove the files in

 

/var/cpanel/datastore/<username>/

 

Then run

 

/scripts/fixmysql

 

You should now the correct information showing in the control panel.

Author: Chris Burton
Last update: 2014-12-16


Can't recover Helm passwords via the website - Reason: Failure sending mail

When using the "Reason: Failure sending mail" option in Helm and you receive the error "An exception occurs while sending the email out, please contact your administrator. Reason: Failure sending mail." this is most likely caused by the mail server not relaying mail for the local IP address without authentication.

 

To resolve the problem in SmarterMail login as the admin user.

Select "Security" and then "SMTP Authentication Bypass" - then using the "Add IP Address(s)" option add the external IP address of the server to the allowed to relay list.

 

You should now be able to recover passwords via the Helm login page.

Author: Chris Burton
Last update: 2014-12-16


Error "invalid maildirsize file" when updating mailbox quota on cPanel.

To resolve the problem "invalid maildirsize file"

Login to your cPanel server using SSH and run

 /scripts/generate_maildirsize <username>

You will need to read the information provided and then re-run the command with the additional option specified to ensure you are aware of what the command is going to do.

Once the command has been correctly ran with the additional parameter you will be able to update the quota within cPanel.

Author: Chris Burton
Last update: 2014-12-16


How do I log into SmarterStats as the admin?

To login into Smarter Stats as the admin user you need to use:

 

Site ID: admin
Username: admin
Password: <your password>

You should now be logged into SmarterStats as the admin user. If you have changed the name of the admin user you will need to use this updated name whilst logging in.

Author: Chris Burton
Last update: 2014-12-16


After un-suspending a site in ensim email is not received.

After unsuspending sites sometimes the /dev/ directory in the virtual sites directory is missing.

----- The following addresses had permanent fatal errors ----- <user@domain.com>
(reason: critical OS file missing)
----- Transcript of session follows -----
procmail: Error while writing to "/dev/null"
554 5.3.5 System file missing

The /dev/ directory can normally be recreated by suspending and then unsuspending the site.

DisableVirtDomain <domainname>
EnableVirtDomain <domainname>

Mail delivery should now continue.

Author: Chris Burton
Last update: 2014-12-16


My Softaculous licence is showing as expired but I've renewed it.

To force Softaculous to re-check the licence you will need to run the cron, for cPanel servers this is as simple as logging into your server as the root user (SSH) and running the following command.

/usr/local/cpanel/3rdparty/bin/php /usr/local/cpanel/whostmgr/docroot/cgi/softaculous/cron.php 

You should them be able to use Softaculous from WHM/cPanel.

Author: Chris Burton
Last update: 2014-12-16


DotNetPanel / WebSitePanel custom errors do not work

If IIS has been configured to show detailed errors by default custom error (404 etc) handlers will not function correctly, to resolve this edit your "web.config" file editing the "httpErrors" start tag to include the "errorMode" parameter as shown below (you will need to add the text into the existing tag) similar to below:

<httpErrors errorMode="DetailedLocalOnly">

When you view your website with this in the web.config file you should see custom errors displayed.

Author: Chris Burton
Last update: 2014-12-16


How do I log the connection IP in qmail (H-Sphere)?

In versions newer than H-Sphere 2.5 Patch 1 you can increase the logging level by editing the "/var/qmail/control/options" file. Simply add (or edit any existing setting) to have "smtplog=2" on it's own line.

 

WIth this added restart qmail "/etc/init.d/qmaild restart" new connections will now be logged will be more verbose - into "/var/log/maillog" this will also include the IP address used to initiate the connection to qmail.

Author: Chris Burton
Last update: 2014-12-16


What is the MySQL password for OnApp installation?

You can view the MySQL database name and password for an onapp in the following file.

cat /onapp/interface/config/database.yml

You can then use this information to connect to the OnApp MySQL database.

Author: Chris Burton
Last update: 2014-12-16


How do I update the email usage in cPanel for a single account?

To update the mailbox usage statistics for a cPanel account simply run the following replacing "<USERNAME>" with the cpanel username.

/scripts/generate_maildirsize --confirm --allaccounts --verbose <USERNAME>

Statistics should then show as updated from within cPanel for the user.

Author: Chris Burton
Last update: 2014-12-16


How do I run a backup for a single account in DirectAdmin?

Set the USER variable to the account name to backup.

USER="example"
echo "action=backup&append%5Fto%5Fpath=nothing&database%5Fdata%5Faware=yes&email%5Fdata%5Faware=yes&local%5Fpath=%2Fhome%2Fadmin%2Fadmin%5Fbackups&owner=admin&select%30=$USER&type=admin&value=multiple&when=now&where=local" >> /usr/local/directadmin/data/task.queue

 

Author: Chris Burton
Last update: 2019-09-30


Internet » Email

I keep getting “Permission denied” errors with snertsoft milters

Updating /etc/mail/access file the milter processes can generate “open error hash … Permission denied” errors.

When the files are automatically updated the permissions on the .db files will be changed back to only allow root to read them, if you are OK with anyone with an account on the box reading the files you can resolve this issue as follows:

Edit the /etc/mail/Makefile and find the section

%.db: %
        @makemap hash $@ < $<

Add a new line after this and put in the following text

        @chmod a+r $@

Ensuring you use a TAB in the Makefile

Now everytime the file is updated the permissions on the db file will be set to allow all users on the box to read the files which includes the milter processes.

Author: Chris Burton
Last update: 2014-12-16


How do I disable Greylisting in SmarterMail?

To disable greylisting (graylisting) you will need to login as an admin and follow these simple steps.

 

  1. Expand “Domains” and click on “All Domains” and find the domain you wish to disable greylisting for.
  2. Right click on the domain and select “Edit”
  3. Select the “Technical” tab
  4. Check the box for “Disable Greylisting”
  5. Click “Save”

Greylisting should now be disabled for this domain.

Author: Chris Burton
Last update: 2014-12-16


How do I test SMTP Auth using telnet?

To test AUTH LOGIN

Prepare the username and password by encoding them as follows:

 

$ perl -MMIME::Base64 -e 'print encode_base64("user\@domain");'
dXNlckBkb21haW4= 
$ perl -MMIME::Base64 -e 'print encode_base64("password");'
 cGFzc3dvcmQ=

 

Please make sure you use a '\' to escape the @ symbol in any username otherwise perl may interpret it.

Now connect to the mail server using telnet

 

telnet mailserver.com 25

 

Once the server has sent a greeting (220) reply as below replacing "example.com" with the hostname of your computer.

 

EHLO example.com

 

The server should then reply back and the AUTH line must include the text LOGIN (i.e. 250-AUTH PLAIN LOGIN).

You now need to tell the server you wish to authenticate with it so send the following.

 

AUTH LOGIN

 

The server should then reply with "334 VXNlcm5hbWU6" (which is "Username:" base64 encoded) and to this paste in the username prepared above for example.

dXNlckBkb21haW4=

The server will again reply this time with "334 UGFzc3dvcmQ6" (which is "Password:" base64 encoded) and to this paste in the password prepared above for example.

cGFzc3dvcmQ=

If the username and password were both accepted you should then see

235 Authentication succeeded

If this is not the reply then you may have a problem with your email credentials.

Author: Chris Burton
Last update: 2014-12-16


How do I block inbound mail to a Linux server?

To temporarily block inbound mail to a mail server by  blocking access to port 25 on the server use the following iptables command (this allows you the mail server process to continue to run so you can sort resolve any problems before allowing access):

iptables -A INPUT -p tcp -d <SERVER_IP> --dport 25 -j REJECT

This assumes your input rule is called INPUT, if you have existing custom chains/rules you may need to update this command to fit your configuration.

To allow mail back into the server delete the rule using:

iptables -D INPUT -p tcp -d 64.22.86.210 --dport 25 -j REJECT

You can view existing rules using

iptables-save

or

iptables -L -n

Author: Chris Burton
Last update: 2014-12-16


Error "invalid maildirsize file" when updating mailbox quota on cPanel.

To resolve the problem "invalid maildirsize file"

Login to your cPanel server using SSH and run

 /scripts/generate_maildirsize <username>

You will need to read the information provided and then re-run the command with the additional option specified to ensure you are aware of what the command is going to do.

Once the command has been correctly ran with the additional parameter you will be able to update the quota within cPanel.

Author: Chris Burton
Last update: 2014-12-16


After un-suspending a site in ensim email is not received.

After unsuspending sites sometimes the /dev/ directory in the virtual sites directory is missing.

----- The following addresses had permanent fatal errors ----- <user@domain.com>
(reason: critical OS file missing)
----- Transcript of session follows -----
procmail: Error while writing to "/dev/null"
554 5.3.5 System file missing

The /dev/ directory can normally be recreated by suspending and then unsuspending the site.

DisableVirtDomain <domainname>
EnableVirtDomain <domainname>

Mail delivery should now continue.

Author: Chris Burton
Last update: 2014-12-16


How do I log the connection IP in qmail (H-Sphere)?

In versions newer than H-Sphere 2.5 Patch 1 you can increase the logging level by editing the "/var/qmail/control/options" file. Simply add (or edit any existing setting) to have "smtplog=2" on it's own line.

 

WIth this added restart qmail "/etc/init.d/qmaild restart" new connections will now be logged will be more verbose - into "/var/log/maillog" this will also include the IP address used to initiate the connection to qmail.

Author: Chris Burton
Last update: 2014-12-16


How do I see queued messages in qmail?

To view a count of the messages in qmail 

 

/var/qmail/bin/qmail-qstat

 

To view more details on the messages in the queue

 

/var/qmail/bin/qmail-qread

 

To view the message content take the # number shown from the commands above and run

 

less /var/qmail/queue/*/<NUMBER>

 

Author: Chris Burton
Last update: 2014-12-16


How do I update the email usage in cPanel for a single account?

To update the mailbox usage statistics for a cPanel account simply run the following replacing "<USERNAME>" with the cpanel username.

/scripts/generate_maildirsize --confirm --allaccounts --verbose <USERNAME>

Statistics should then show as updated from within cPanel for the user.

Author: Chris Burton
Last update: 2014-12-16


Software » Linux

How do I stop an automatic raid check?

MD (software) raid checks are performed every Sunday at 1am on Centos6 systems, to stop a current raid check you can run the following command (updating the raid device number to that of your device).

echo "idle" > /sys/block/md0/md/sync_action

When you now view the status of the software RAID device you should see the sync has stopped.

cat /proc/mdstat

It may have moved onto anothe device if this was delayed due to it sharing a physical device with the previous check - if this is the case you may also want to stop this check.

 

N.B. stopping these checks may lead to data loss if your raid device needs to be checked or has a fault.

Author: Chris Burton
Last update: 2014-12-16


How do I stop X from starting on boot on the Iteaduino Plus?

From the console you can disable X at startup by issuing the following command.

sudo mv /etc/init/lightdm.conf /etc/init/lightdm.conf.disabled

And then reboot the board.

You will then find you no longer have the X graphical system running and an extra ~100MB of memory to use.

If you wish to re-enable X at startup run the following to "undo" it.

sudo mv /etc/init/lightdm.conf.disabled /etc/init/lightdm.conf

And again following a reboot the service will be re-enabled.

Author: Chris Burton
Last update: 2014-12-16


How do I set/reset the linaro users password?

If you haven't previously changed the linaro users password the default is "linaro", so you can reset the password by issuing the following command.

passwd

And enter the old password and then repeat the new password to set your new password.

If you do not currently know the password (but have been automatically logged in) you can reset the password via

sudo passwd linaro

This will allow you to reset the password without knowing the existing password.

Author: Chris Burton
Last update: 2014-12-16


How do I recompile the kernel (and install those missing modules) on the Iteaduino Plus?

Install the required packages to recompile the kernel with (logged in as root) and prepare the directory for the source code.

 

mkdir ~/installed
cd ~/installed
apt-get install build-essential git debootstrap u-boot-tools libncurses5-dev

 

Download the tools needed to compile the kernel and configure the environment:

 

git clone https://github.com/linux-sunxi/sunxi-bsp.git
cd sunxi-bsp/scripts
export PATH=$PATH:$PWD
cd ../..

 

Author: Chris Burton
Last update: 2014-12-16


How do I setup CentOS 7 for KVM Virtualisation with a bridged network [OVH/Hetzner]?

Coming from a minimal install the first step is to install the required software

# yum install net-tools qemu-kvm libvirt virt-install bridge-utils bind-utils

Convert the network configuration to a static IP address

Both Hetzner and OVH supply their dedicated servers were the network is configured via DHCP, this needs to be updated to a static configuration for the bridged network for KVM to work correctly.

To view the current network configuration run.

# ifconfig

With CentOS 7 the days of ethX are gone (unless it's OVH), they will now be labelled something like enp6s0

enp6s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.10.10.10  netmask 255.255.255.224  broadcast 176.9.36.255
        inet6 fe80::8e89:a5ff:fe63:b8ef  prefixlen 64  scopeid 0x20<link>
        ether 8c:89:a5:63:b8:ef  txqueuelen 1000  (Ethernet)
        RX packets 69106191  bytes 7806556849 (7.2 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 197727632  bytes 274850510321 (255.9 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

You will also need the gateway to setup the network.

# ip route

The output should have line prefixed with "default via".

default via 10.10.10.1 dev enp6s0  proto static  metric 1024

In the example above the default gateway is 10.10.10.1.

Replacing the name of your network device create a backup of the current network configuration (this must be a PREFIX otherwise it will be treated as a network device and will be configured on boot causing issues).

# cp /etc/sysconfig/network-scripts/ifcfg-enp6s0 /etc/sysconfig/network-scripts/bak.ifcfg-enp6s0

 Edit the configuration file (again replacing your network device name as required).

# vi /etc/sysconfig/network-scripts/ifcfg-enp6s0

You should see something like the following

# Generated by dracut initrd
DEVICE="enp6s0"
ONBOOT=yes
NETBOOT=yes
UUID="d2183429-2b16-43b1-99d6-433cf3c386ab"
IPV6INIT=yes
BOOTPROTO=dhcp
HWADDR="8c:89:a5:61:a8:ef"
TYPE=Ethernet
NAME="enp6s0"
DNS1="213.133.99.99"

 This needs to be changed to look like the following (dns shown are for Hetzner).

DEVICE="enp6s0"
ONBOOT=yes
NETBOOT=no
NM_CONTROLLED="no"
BOOTPROTO=static
HWADDR="8c:89:a5:61:a8:ef"
TYPE=Ethernet
NAME="enp6s0"
DNS1="213.133.98.98"
DNS2="213.133.99.99"
DNS3="213.133.100.100"
IPADDR=10.10.10.10
NETMASK=255.255.255.224
GATEWAY=10.10.10.1

Replacing the IPADDR/NETMASK with the inet initially shown on the network device, the GATEWAY IP from the output of  the "ip route" command.

Initially I would try restarting the network without a reboot 

# service network restart

If the network has been configured correctly the command prompt should return. If the command prompt doesn't come back you will need to boot the server in recovery mode and then either revert the changes from the bak file or correct any errors.

I would also advise doing reboot to ensure the network will start up like this as this is the basis of the network used later.

# reboot

Again if the server comes back after the reboot all is good, otherwise you will need to boot into recovery and resolve the issue.

Setup libvirt and configure the bridge

Start and enable libvirtd to start at system boot.

# systemctl start libvirtd
# systemctl enable libvirtd

Again we now need to copy the network file (changing the filename to your device as required) to make setting up the bridge network configration a little easier.

# cp /etc/sysconfig/network-scripts/ifcfg-enp6s0 /etc/sysconfig/network-scripts/ifcfg-br0

Edit the config for the network device.

# vi /etc/sysconfig/network-scripts/ifcfg-enp6s0

Comment out the HWADDR, TYPE, NAME, DNS1/DNS2/DNS3, IPADDR, NETMASK and GATEWAY and add the BRIDGE line which should leave you with something like this (we are calling the bridge to use with KVM "br0").

DEVICE="enp6s0"
ONBOOT=yes
NETBOOT=no
NM_CONTROLLED="no"
#BOOTPROTO=static
#HWADDR="8c:89:a5:61:a8:ef"
#TYPE=Ethernet
#NAME="enp6s0"
#DNS1="213.133.98.98"
#DNS2="213.133.99.99"
#DNS3="213.133.100.100"
#IPADDR=10.10.10.10
#NETMASK=255.255.255.224
#GATEWAY=10.10.10.1
BRIDGE=br0

 And then edit the new file for the bridge configuration.

# vi /etc/sysconfig/network-scripts/ifcfg-br0

After updating the DEVICE, TYPE, NAME and comment out the GATEWAY you should have something like.

DEVICE="br0"
ONBOOT=yes
NM_CONTROLLED="no"
NETBOOT=no
IPV6INIT=yes
BOOTPROTO=static
#HWADDR="8c:89:a5:63:b8:ef"
TYPE=Bridge
NAME="br0"
DNS1="213.133.99.99"
IPADDR=10.10.10.10
NETMASK=255.255.255.224
#GATEWAY=10.10.10.1

You will now need to add the commented out GATEWAY line to your network configuration

# vi /etc/sysconfig/network

And add the removed GATEWAY line

GATEWAY=10.10.10.1

Once these fines have been updated and checked restart the network service

# service network restart

Now check your new bridge has the correct configuration with the following commans.

# ifconfig br0
# ifconfig enp6s0

The br0 device should now have the IP address and the enp6s0 device should have no IP address associated with it.

Once you are happy with the configuration reboot the server.

# reboot

If everything looks OK after reboot you can now proceed with your KVM system and setup new Virtual Servers (with virt-manager for example). If your server doesn't come back after a few minutes you will need to boot into recovery mode and check the configuration files.

Author: Chris Burton
Last update: 2018-05-05


How do I change the timezone on Centos 7 to UTC?

CentOS 7 now includes a tool to change the timezone, to view your current timezone run the following command.

timedatectl

This should display the current timezone configuraton. To change this to UTC (or to any other timezone) run the following.

timedatectl set-timezone UTC

You can then re-check the timezone by running the command above again.

Author: Chris Burton
Last update: 2015-01-03


How do I setup SNMPd on CentOS 7?

To setup SNMP daemon firstly install the required packages.

yum install net-snmp net-snmp-devel net-snmp-libs net-snmp-utils

Then edit the main configuration file

vi /etc/snmp/snmpd.conf

Find the line starting with "com2sec" and change the community from "public" to your secret v2c community name. If you also wish to use version v1 you can also add a line "rocommunity SECRET_COMMUNITY" at the end of the file.

You will then need to start the service

service snmpd start

And configure it to start automatically

systemctl enable snmpd

The SNMP daemon should now be running which can be used to monitor the server via Cacti etc.

 

 

Author: Chris Burton
Last update: 2015-01-14


How do I install the same RPM packages on a new server as an old?

First you need a list of the packages on the old server.

rpm -qa --qf "%{NAME}\n" > rpmlist.txt

 

Copy this file to the new server (for example using SSH).

 

To bulk install all of the packages installed on the old server

yum install `cat rpmlist.txt|tr '\n' ' '`

 

You should then be prompted to install all packages missing on the new server.

Author: Chris Burton
Last update: 2015-08-24


How do I get the disk usage of a directory including hidden directories/files?

Using du with a wildcard would not normally include "hidden" directories/files (those starting with a dot). But you can show the usage with this simple command.

du -sch .[!.]* *

 

Author: Chris Burton
Last update: 2015-08-24


How do I alter the kernel cmdline and rebuild grub2 on CentOS 7?

 

The kernel command line (cmdline) can be edited in "/etc/default/grub" (GRUB_CMDLINE_LINUX).

 

Once edited you will need to rebuild the config for grub2 using the following command

 

grub2-mkconfig --output=/boot/grub2/grub.cfg

 

You can then reboot and the new cmdline will be used (this can be verified by "cat /proc/cmdline").

 

 

Author: Chris Burton
Last update: 2016-09-15


How do I show disk usage breakdown including hidden directories?

Using "du" with the following options will show usage of all files and directories including hidden (those starting with a dot).

du -sch .[!.]* *

Author: Chris Burton
Last update: 2018-08-30


How do I resize a disk image (Raspbian SD/USB image)?

To resize the last partition in a disk image, for example a Raspberry Pi SD card image first increase the size of the file on disk using truncate, here we're going to increase it by 500MB.

truncate 2019-06-20-raspbian-buster-full.img --size=+500M

Once the file size has been increased the size of the second partition need to be updated to use all of the remaining space.

parted --script ./2019-06-20-raspbian-buster-full.img resizepart 2 100%

And then the filesystem will need to be extended to the size of the new partition, first enable access to the partitions in the disk image.

losetup -fP --show 2019-06-20-raspbian-buster-full.img

This will return the loop device name, in this example we're going to assume it's "/dev/loop1". 

Then check the filesystem.

e2fsck -f /dev/loop1p2

Now we can do the actual filesystem resize.

resize2fs /dev/loop1p2

If you see no errors then we're done and we can remove the loop interface.

losetup -d /dev/loop1

The size of the disk image has now been extended and the second partition has been increased to use all of the space.

Author: Chris Burton
Last update: 2019-07-24


When using CNAT the br0 (controller) or usb0 (node) doesn't get an IP via DHCP or set the fallback IP.

From testing I have seen this happen when dhcpcd is unable to obtain an IP address for the interface before systemd classes it as a timeout and kills the service.

This can be fixed by increasing the time systemd lets the service run before terminating it.

sudo systemctl edit --full dhcpcd5.service

After the last line in the [Service] section append "TimeoutStartSec=900" and save the file and then reload the changes with.

sudo systemctrl daemon-reload

After rebooting you should now either get an IP via DHCP or use the fallback IP.

Author: Chris Burton
Last update: 2019-08-03


Software » Linux » Command Line

I keep getting “Permission denied” errors with snertsoft milters

Updating /etc/mail/access file the milter processes can generate “open error hash … Permission denied” errors.

When the files are automatically updated the permissions on the .db files will be changed back to only allow root to read them, if you are OK with anyone with an account on the box reading the files you can resolve this issue as follows:

Edit the /etc/mail/Makefile and find the section

%.db: %
        @makemap hash $@ < $<

Add a new line after this and put in the following text

        @chmod a+r $@

Ensuring you use a TAB in the Makefile

Now everytime the file is updated the permissions on the db file will be set to allow all users on the box to read the files which includes the milter processes.

Author: Chris Burton
Last update: 2014-12-16


Where can I find the MySQL root user password for DirectAdmin?

How do I get mysqls root user password for DirectAdmin?

The username and password used to access MySQL as the “root” user can be obtained from the following file.

/usr/local/directadmin/conf/mysql.conf

In this file you will see the current “root” username and password used by DirectAdmin to access the database.

Author: Chris Burton
Last update: 2014-12-16


How do I convert text in a file to lower case?

Converting the content of a file to lowercase a simple process using ‘tr’

cat input.txt | tr ‘[:upper:]‘ ‘[:lower:]‘ > output.txt

Remember the input.txt and output.txt can not be the same file.

To change a file to all uppercase just swap upper and lower.

Author: Chris Burton
Last update: 2014-12-16


How do I recalculate Plesk statistics for a site?

If for some reason Plesk does not update the statistics properly for a site or you wish to re-run them after changing log files etc it is a simple process.

 

  1. Login to your Plesk server as user “root” using SSH.
  2. Run “/usr/local/psa/admin/sbin/statistics --calculate-one --domain-name=<domainname>” – replacing <domainname> with the domain name you need to reprocess the statistics for.

Author: Chris Burton
Last update: 2014-12-16


How do I fix Unable to connect to database: saved admin password is incorrect (Plesk)?

ERROR: PleskFatalException
Unable to connect to database: saved admin password is incorrect.

0: /usr/local/psa/admin/plib/common_func.php3:190
psaerror(string ‘Unable to connect to database: saved admin password is
incorrect.’)
1: /usr/local/psa/admin/auto_prepend/auth.php3:93

The following instructions assume your Plesk admin user is “admin” and you with to reset the password to “setup”. You can check the password Plesk is expecting to use by doing “cat /etc/psa/.psa.shadow”.

  1. Using your favoured editor create the file “/root/mysqlreset.sql” with the content “UPDATE mysql.user SET Password=PASSWORD(’setup’) WHERE User=’admin’;FLUSH PRIVILEGES;”.
  2. Stop the mysql server “/etc/init.d/mysqld stop” (this will cause issues with any sites utilising MySQL).
  3. Start the MySQL server using the initial SQL file created above “mysqld_safe –init-file=/root/mysqlreset.sql &”.
  4. This should have reset the password and you can now restart MySQL.

You should now be able to login to the Plesk control panel via http://<sitename>:8443/.

Author: Chris Burton
Last update: 2014-12-16


How do I graphing mysql statistics in Cacti?

Graphing mysql statistics in Cacti

The HOWTO below will guide you through creating statistics graphs for MySQL usage in Cacti.

 

We have modified the authentication to use the standard mtop user which was already on our system.

I saved this script as /usr/local/bin/cacti-mysqlstats.php and set it executable with:

chmod a+rx /usr/local/bin/cacti-mysqlstats.php

Here is a local cached copy of the script from: http://www.cacti.net/additional_scripts.php [Gets statistics from MySQL server]
#!/usr/bin/php -q
<?

/* sql_stat – Gets statistics from MySQL server

Takes one argument (Variable name) and returns it’s value.
Good replacement for the sql.php script.

some interesting variables:

Questions – queries executed since last restart.
Threads_running – running queries now
Threads_connected – connected clients
.
.
For more info RTFM or mail me:

Martin Spasov (mspasov@techno-link.com)
This script is distributed under GPL.

*/

$var = $argv[1];
mysql_connect(“localhost”, “mysqltop”) or die (“0″);
$res = mysql_query(“SHOW STATUS”) or die(“0″);
while ($a = mysql_fetch_row($res)) {
$stat[$a[0]] = $a[1];
}
printf(“%s”, $stat[$var]);
mysql_close();
?>

Data Input Method

To graph the data we need an input method

  1. Click – Data Input Methods (left hand menu)

To graph Queries

  1. Click – Add (top right hand corner)
  2. Should now show “Data Input Methods [new]“
  3. Set: Name -> Mysql – “Queries”
  4. Set: Input Type -> Query/Command
  5. Set: Input String -> “/usr/local/bin/cacti-mysqlstats.php Queries”
  6. Click – Create
  7. Click Output Fields -> Add
  8. Set: Field [Output] -> “queries”
  9. Set: Friendly Name -> “Queries”
  10. Click – Create
  11. Click – Save

Data Templates

  1. Click – Data Templates

To graph Queries

  1. Click – Add (top right hand corner)
  2. Set: Data Templates [new] -> “Mysql – Queries”
  3. Tick: Data Source “Use Per-Data Source Value (Ignore this Value)”
  4. Set: Data Input Method -> Mysql – Queries

Author: Chris Burton
Last update: 2014-12-16


How do I test SMTP Auth using telnet?

To test AUTH LOGIN

Prepare the username and password by encoding them as follows:

 

$ perl -MMIME::Base64 -e 'print encode_base64("user\@domain");'
dXNlckBkb21haW4= 
$ perl -MMIME::Base64 -e 'print encode_base64("password");'
 cGFzc3dvcmQ=

 

Please make sure you use a '\' to escape the @ symbol in any username otherwise perl may interpret it.

Now connect to the mail server using telnet

 

telnet mailserver.com 25

 

Once the server has sent a greeting (220) reply as below replacing "example.com" with the hostname of your computer.

 

EHLO example.com

 

The server should then reply back and the AUTH line must include the text LOGIN (i.e. 250-AUTH PLAIN LOGIN).

You now need to tell the server you wish to authenticate with it so send the following.

 

AUTH LOGIN

 

The server should then reply with "334 VXNlcm5hbWU6" (which is "Username:" base64 encoded) and to this paste in the username prepared above for example.

dXNlckBkb21haW4=

The server will again reply this time with "334 UGFzc3dvcmQ6" (which is "Password:" base64 encoded) and to this paste in the password prepared above for example.

cGFzc3dvcmQ=

If the username and password were both accepted you should then see

235 Authentication succeeded

If this is not the reply then you may have a problem with your email credentials.

Author: Chris Burton
Last update: 2014-12-16


How do I install Softaculous on cPanel?

To install Sofaculous follow these simple instructions:

  1. Login via SSH to the server (as root, or "su - root" if accessing via another user.
  2. Execute the following commands.
    cd /usr/local/cpanel/whostmgr/docroot/cgi
    wget -N http://www.softaculous.com/ins/addon_softaculous.php
    chmod 755 addon_softaculous.php
     
  3. Login to WHM.
  4. Under "Server Configuration" click on "Tweak Settings".
  5. Under the "PHP" heading enable "ioncube" for "Loader to use for internal cPanel PHP".
  6. Click "Save".
  7. Refresh the menu in WHM (normally by pressing F5).
  8. In the "Plugins" menu select "Softaculous - Instant Installs".
  9. Softaculous will now download packages - wait for this to complete (scroll within the frame to check).
  10. Once completed click on "Software" to view the installed software.
  11. If you using the Free licence you can now use Softaculous otherwise continue with the next step.
  12. To upgrade the Licence to Premium click on "Home" under the Softaculous banner.
  13. Click on "Buy Premium License" - you will now be taken to the Sofaculous website.
  14. Make a note of the "Softaculous License" number.
  15. Click on "Buy a license".
  16. Login to Softaculous to proceed with the order.
  17. Click on "My Licenses" and open in a new window.
  18. Enter the license number noted above and "Add License".
  19. Close the window and return to the main Softaculous site.
  20. Enter in the License number noted above and click "Purchase Softaculous".
  21. Select your payment method and make the payment.
  22. In the "Plugins" menu of WHM select "Softaculous - Instant Installs".
  23. Click "Refresh License" which should then show the Premium Licence.
  24. Click the "Software" heading.
  25. Check the box just below "Installed" (this will check all boxes below) and click "Update Settings".

Once the above steps have been completed you will have a fully installed and updated Premium installation of Softaculous.

Author: Chris Burton
Last update: 2014-12-16


How do I block inbound mail to a Linux server?

To temporarily block inbound mail to a mail server by  blocking access to port 25 on the server use the following iptables command (this allows you the mail server process to continue to run so you can sort resolve any problems before allowing access):

iptables -A INPUT -p tcp -d <SERVER_IP> --dport 25 -j REJECT

This assumes your input rule is called INPUT, if you have existing custom chains/rules you may need to update this command to fit your configuration.

To allow mail back into the server delete the rule using:

iptables -D INPUT -p tcp -d 64.22.86.210 --dport 25 -j REJECT

You can view existing rules using

iptables-save

or

iptables -L -n

Author: Chris Burton
Last update: 2014-12-16


How do I get a list of all databases on pgsql/PostgreSQL?

To get a list of all databases on PostgreSQL su to the postgres user:

 

su - postgres

 

Once you have switched to the postgres user simply issue the following command to list all databases.

 

psql -l

You should now see the full list of PostgreSQL databases on this system.

 

Author: Chris Burton
Last update: 2014-12-16


How do I backup a pgsql/PostgreSQL database?

To get a list of all databases on PostgreSQL su to the postgres user:

 

su - postgres

 

Once you have switched to the postgres user simply issue the following command to backup the database.

 

pg_dump <database> > <databasebackup.sql>

You should now have a complete backup of the database content.

Author: Chris Burton
Last update: 2014-12-16


How do I update the Plesk limits when the user is over disk quota?

Normally the Plesk control panel will not allow you to update the limits of a hosting account if the account is over quota. This can quite easily be achieved by altering the values stored in the database whilst updating the account.

 

Login to your Plesk server and connect to the MySQL server (psa database).

 

mysql -u admin -p`cat /etc/psa/.psa.shadow` psa

 

Once connected check (and keep) the current disk usage value (replace <DOMAINNAME>).

 

SELECT real_size FROM domains WHERE name='<DOMAINNAME>';

 

You will now see something similar to this - record the value given

 

+-----------+
| real_size |
+-----------+
| 434324252 |
+-----------+

 

Update the value to a lower amount which will allow you to update the limits within Plesk.

 

UPDATE domains SET real_size=0 WHERE name='<DOMAINNAME>';

 

Login to your Plesk control panel and update the account limits.

Once the limits have successfully been updated in Plesk you can restore the old disk usage value (replacing the number for the real_size with that shown in the SELECT query and again replacing <DOMAINNAME>).

 

UPDATE domains SET real_size=434324252  WHERE name='<DOMAINNAME>';

This completes the process and the Plesk control panel should now show the new Limits with the "over quota" disk usage shown correctly.

 

 

Author: Chris Burton
Last update: 2014-12-16


Why is cPanel is showing disk usage when the account has no database?

Sometimes cPanel does not remove the database correctly and leaves behind files.

 

Take a backup and then remove the files in

 

/var/cpanel/datastore/<username>/

 

Then run

 

/scripts/fixmysql

 

You should now the correct information showing in the control panel.

Author: Chris Burton
Last update: 2014-12-16


How do I view the expiry date of an SSL certificate?

To view the expiry dates of an SSL certificate simply run

openssl x509 -noout -in <filename> -dates

Which will output data in a format similar to this

notBefore=Jan  1 00:00:00 2011 GMT
notAfter=Jan  1 23:59:59 2012 GMT

If you wish to view all data stored in the certificate file you can run this which will show more details (and also includes the expiry date).

openssl x509 -noout -in <filename> -text

Author: Chris Burton
Last update: 2014-12-16


How do I convert a .asf file to .wav?

Using mplayer it is simple to convert an ASF file to MP3.

Ensure you have mplayer installed (On CentOS this would be "yum install mplayer").

mplayer infile.asf -vo null -ao pcm:waveheader:file=outfile.wav

This will then convert the audio file into a .wav for further processing, etc.

Author: Chris Burton
Last update: 2014-12-16


How do I list all packages and versions on Ubuntu?

To list installaed software and the versions numbers first install apt-show-versions

sudo apt-get install apt-show-versions

They run apt-show-versions which will display currently installed software and versions.

Author: Chris Burton
Last update: 2014-12-16


Error "invalid maildirsize file" when updating mailbox quota on cPanel.

To resolve the problem "invalid maildirsize file"

Login to your cPanel server using SSH and run

 /scripts/generate_maildirsize <username>

You will need to read the information provided and then re-run the command with the additional option specified to ensure you are aware of what the command is going to do.

Once the command has been correctly ran with the additional parameter you will be able to update the quota within cPanel.

Author: Chris Burton
Last update: 2014-12-16


How do I remove all lines from a file up to a keyword?

To remove all lines up to and including a keyword from a file simply use sed as follows:

sed '1,/keyword/d' infile > outfile

(replacing keyword/infile/outfile with the relevant data)

Author: Chris Burton
Last update: 2014-12-16


After un-suspending a site in ensim email is not received.

After unsuspending sites sometimes the /dev/ directory in the virtual sites directory is missing.

----- The following addresses had permanent fatal errors ----- <user@domain.com>
(reason: critical OS file missing)
----- Transcript of session follows -----
procmail: Error while writing to "/dev/null"
554 5.3.5 System file missing

The /dev/ directory can normally be recreated by suspending and then unsuspending the site.

DisableVirtDomain <domainname>
EnableVirtDomain <domainname>

Mail delivery should now continue.

Author: Chris Burton
Last update: 2014-12-16


My Softaculous licence is showing as expired but I've renewed it.

To force Softaculous to re-check the licence you will need to run the cron, for cPanel servers this is as simple as logging into your server as the root user (SSH) and running the following command.

/usr/local/cpanel/3rdparty/bin/php /usr/local/cpanel/whostmgr/docroot/cgi/softaculous/cron.php 

You should them be able to use Softaculous from WHM/cPanel.

Author: Chris Burton
Last update: 2014-12-16


How do I log the connection IP in qmail (H-Sphere)?

In versions newer than H-Sphere 2.5 Patch 1 you can increase the logging level by editing the "/var/qmail/control/options" file. Simply add (or edit any existing setting) to have "smtplog=2" on it's own line.

 

WIth this added restart qmail "/etc/init.d/qmaild restart" new connections will now be logged will be more verbose - into "/var/log/maillog" this will also include the IP address used to initiate the connection to qmail.

Author: Chris Burton
Last update: 2014-12-16


After upgrading Cacti all image/css/etc links are pointing to / rather than /cacti/

After upgrading Cacti from RPMForge cacti images/etc are linked to / rather than the /cacti/ directory.

To resolve this you will need to edit the "/var/www/cacti/include/config.php" file will your chosen text editor and add in the following config option.

$url_path = "/cacti/";

After this has been saved you will be able to visit your Cacti installation and the links will function correctly.

Author: Chris Burton
Last update: 2014-12-16


What is the MySQL password for OnApp installation?

You can view the MySQL database name and password for an onapp in the following file.

cat /onapp/interface/config/database.yml

You can then use this information to connect to the OnApp MySQL database.

Author: Chris Burton
Last update: 2014-12-16


How do I find the serial/model number of a hard drive?

To find the model or serial number of a hard drive you can use hdparm and the device name of the hard drive you need the serial number of.

hdparm -i /dev/sda

Replacing "/dev/sda" with the device of the hard drive the information is needed for which will return data as follows.

Model=<model>, FwRev=<revision>, SerialNo=<serial>

Author: Chris Burton
Last update: 2014-12-16


How do I copy the partition layout to a new drive?

It's useful in RAID setups to copy the partition tables over to a second/third/etc drive to save using fdisk to set them all up. To copy the partition table layout from /dev/sda to /dev/sdb you would use the following command.

sfdisk -d /dev/sda | sfdisk /dev/sdb

Ensure you are using the correct source (/dev/sda above) and destination (/dev/sdb above) devices before issuing the command.

Author: Chris Burton
Last update: 2014-12-16


How do I shutdown/stop an alien software RAID device?

When using a tool suck as SysRescCD to access data on alien drives you may find you need to disable the RAID (i.e. to allow you to do a block level copy).

First you need the system to recognise the RAID devices.

mdadm --examine --scan > /etc/mdam.conf

(ensure you only use this on a SysRescCD and not on a live server otherwise your exisint gfile will be modified)

You will now be able to stop/start the raid devices as normal

mdadm --stop /dev/md0

Author: Chris Burton
Last update: 2014-12-16


How do I see the progress of a dd?

To view the current position of a dd you need to send signal SIGUSR1

You can either find the pid of dd and send the signal with

kill -SIGUSR1 <PID>

Or if you are only running one dd on the system you can send to all dd processes with

killall -SIGUSR1 dd

Author: Chris Burton
Last update: 2014-12-16


Citrix errror when exporting VM - Export failed due to a block checksum mismatch. Please retry the export

The error "Export failed due to a block checksum mismatch. Please retry the export." can normally be resolved by rebooting the Citrix VM host.

Author: Chris Burton
Last update: 2014-12-17


How do I see queued messages in qmail?

To view a count of the messages in qmail 

 

/var/qmail/bin/qmail-qstat

 

To view more details on the messages in the queue

 

/var/qmail/bin/qmail-qread

 

To view the message content take the # number shown from the commands above and run

 

less /var/qmail/queue/*/<NUMBER>

 

Author: Chris Burton
Last update: 2014-12-16


How do I set/reset the Iteaduino Plus root users password?

When logged in as "linaro" you can reset the root password with the following command

 

sudo password

You will then be able to enter a new root password, you will then be able to login directly via SSH/etc. as the root user.

 

Author: Chris Burton
Last update: 2014-12-16


How do I set/reset the linaro users password?

If you haven't previously changed the linaro users password the default is "linaro", so you can reset the password by issuing the following command.

passwd

And enter the old password and then repeat the new password to set your new password.

If you do not currently know the password (but have been automatically logged in) you can reset the password via

sudo passwd linaro

This will allow you to reset the password without knowing the existing password.

Author: Chris Burton
Last update: 2014-12-16


How do I disable NetworkManager?

If you want to configure your network devices manually via /etc/network/interfaces you can remove the service. Firtstly configure your network devices in /etc/network/interfaces and then issue the following command to remove the service.

sudo apt-get purge network-manager

You will then be able to reboot and not have NetworkManager fiddle with things.

Author: Chris Burton
Last update: 2014-12-16


How do I recompile the kernel (and install those missing modules) on the Iteaduino Plus?

Install the required packages to recompile the kernel with (logged in as root) and prepare the directory for the source code.

 

mkdir ~/installed
cd ~/installed
apt-get install build-essential git debootstrap u-boot-tools libncurses5-dev

 

Download the tools needed to compile the kernel and configure the environment:

 

git clone https://github.com/linux-sunxi/sunxi-bsp.git
cd sunxi-bsp/scripts
export PATH=$PATH:$PWD
cd ../..

 

Author: Chris Burton
Last update: 2014-12-16


How do I find files sorted by last modified time?

 

 

find . -printf "%CY%Cm%Cd-%CI%CM%CS %h/%f\n"|sort -n

Author: Chris Burton
Last update: 2014-12-16


How do I find files sorted by last modified time?

 

 

find . -printf "%CY%Cm%Cd-%CI%CM%CS %h/%f\n"|sort -n

Author: Chris Burton
Last update: 2014-12-16


How do I update the email usage in cPanel for a single account?

To update the mailbox usage statistics for a cPanel account simply run the following replacing "<USERNAME>" with the cpanel username.

/scripts/generate_maildirsize --confirm --allaccounts --verbose <USERNAME>

Statistics should then show as updated from within cPanel for the user.

Author: Chris Burton
Last update: 2014-12-16


How do I setup CentOS 7 for KVM Virtualisation with a bridged network [OVH/Hetzner]?

Coming from a minimal install the first step is to install the required software

# yum install net-tools qemu-kvm libvirt virt-install bridge-utils bind-utils

Convert the network configuration to a static IP address

Both Hetzner and OVH supply their dedicated servers were the network is configured via DHCP, this needs to be updated to a static configuration for the bridged network for KVM to work correctly.

To view the current network configuration run.

# ifconfig

With CentOS 7 the days of ethX are gone (unless it's OVH), they will now be labelled something like enp6s0

enp6s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.10.10.10  netmask 255.255.255.224  broadcast 176.9.36.255
        inet6 fe80::8e89:a5ff:fe63:b8ef  prefixlen 64  scopeid 0x20<link>
        ether 8c:89:a5:63:b8:ef  txqueuelen 1000  (Ethernet)
        RX packets 69106191  bytes 7806556849 (7.2 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 197727632  bytes 274850510321 (255.9 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

You will also need the gateway to setup the network.

# ip route

The output should have line prefixed with "default via".

default via 10.10.10.1 dev enp6s0  proto static  metric 1024

In the example above the default gateway is 10.10.10.1.

Replacing the name of your network device create a backup of the current network configuration (this must be a PREFIX otherwise it will be treated as a network device and will be configured on boot causing issues).

# cp /etc/sysconfig/network-scripts/ifcfg-enp6s0 /etc/sysconfig/network-scripts/bak.ifcfg-enp6s0

 Edit the configuration file (again replacing your network device name as required).

# vi /etc/sysconfig/network-scripts/ifcfg-enp6s0

You should see something like the following

# Generated by dracut initrd
DEVICE="enp6s0"
ONBOOT=yes
NETBOOT=yes
UUID="d2183429-2b16-43b1-99d6-433cf3c386ab"
IPV6INIT=yes
BOOTPROTO=dhcp
HWADDR="8c:89:a5:61:a8:ef"
TYPE=Ethernet
NAME="enp6s0"
DNS1="213.133.99.99"

 This needs to be changed to look like the following (dns shown are for Hetzner).

DEVICE="enp6s0"
ONBOOT=yes
NETBOOT=no
NM_CONTROLLED="no"
BOOTPROTO=static
HWADDR="8c:89:a5:61:a8:ef"
TYPE=Ethernet
NAME="enp6s0"
DNS1="213.133.98.98"
DNS2="213.133.99.99"
DNS3="213.133.100.100"
IPADDR=10.10.10.10
NETMASK=255.255.255.224
GATEWAY=10.10.10.1

Replacing the IPADDR/NETMASK with the inet initially shown on the network device, the GATEWAY IP from the output of  the "ip route" command.

Initially I would try restarting the network without a reboot 

# service network restart

If the network has been configured correctly the command prompt should return. If the command prompt doesn't come back you will need to boot the server in recovery mode and then either revert the changes from the bak file or correct any errors.

I would also advise doing reboot to ensure the network will start up like this as this is the basis of the network used later.

# reboot

Again if the server comes back after the reboot all is good, otherwise you will need to boot into recovery and resolve the issue.

Setup libvirt and configure the bridge

Start and enable libvirtd to start at system boot.

# systemctl start libvirtd
# systemctl enable libvirtd

Again we now need to copy the network file (changing the filename to your device as required) to make setting up the bridge network configration a little easier.

# cp /etc/sysconfig/network-scripts/ifcfg-enp6s0 /etc/sysconfig/network-scripts/ifcfg-br0

Edit the config for the network device.

# vi /etc/sysconfig/network-scripts/ifcfg-enp6s0

Comment out the HWADDR, TYPE, NAME, DNS1/DNS2/DNS3, IPADDR, NETMASK and GATEWAY and add the BRIDGE line which should leave you with something like this (we are calling the bridge to use with KVM "br0").

DEVICE="enp6s0"
ONBOOT=yes
NETBOOT=no
NM_CONTROLLED="no"
#BOOTPROTO=static
#HWADDR="8c:89:a5:61:a8:ef"
#TYPE=Ethernet
#NAME="enp6s0"
#DNS1="213.133.98.98"
#DNS2="213.133.99.99"
#DNS3="213.133.100.100"
#IPADDR=10.10.10.10
#NETMASK=255.255.255.224
#GATEWAY=10.10.10.1
BRIDGE=br0

 And then edit the new file for the bridge configuration.

# vi /etc/sysconfig/network-scripts/ifcfg-br0

After updating the DEVICE, TYPE, NAME and comment out the GATEWAY you should have something like.

DEVICE="br0"
ONBOOT=yes
NM_CONTROLLED="no"
NETBOOT=no
IPV6INIT=yes
BOOTPROTO=static
#HWADDR="8c:89:a5:63:b8:ef"
TYPE=Bridge
NAME="br0"
DNS1="213.133.99.99"
IPADDR=10.10.10.10
NETMASK=255.255.255.224
#GATEWAY=10.10.10.1

You will now need to add the commented out GATEWAY line to your network configuration

# vi /etc/sysconfig/network

And add the removed GATEWAY line

GATEWAY=10.10.10.1

Once these fines have been updated and checked restart the network service

# service network restart

Now check your new bridge has the correct configuration with the following commans.

# ifconfig br0
# ifconfig enp6s0

The br0 device should now have the IP address and the enp6s0 device should have no IP address associated with it.

Once you are happy with the configuration reboot the server.

# reboot

If everything looks OK after reboot you can now proceed with your KVM system and setup new Virtual Servers (with virt-manager for example). If your server doesn't come back after a few minutes you will need to boot into recovery mode and check the configuration files.

Author: Chris Burton
Last update: 2018-05-05


How do I setup VNC for a user on Centos 7?

To setup your server to run a full desktop VNC session first we need to install the required packages.

yum groupinstall "GNOME Desktop";yum install tigervnc-server xorg-x11-fonts-Type1

Once you have the software installed copy the default configuration to the new service (which will run on port 5900 (:0).

cp /lib/systemd/system/vncserver@.service /etc/systemd/system/vncserver@:0.service

Now edit the new file

vi /etc/systemd/system/vncserver@:0.service

And update the 2 occurances of <USER> with your username.

If you are still running the "firewall" service you can add allow inbound connections to the VNC service by using the following commands.

firewall-cmd --permanent --zone=public --add-port=5900/tcp
firewall-cmd --reload

Su to the user and set a VNC password

su - <USER>
vncpasswd
exit # exit back to root user

Configure the service to start the VNC service on boot.

systemctl daemon-reload
systemctl start vncserver@:0.service
systemctl enable vncserver@:0.service

You should now be able to connect to your new VNC service running as your user by using just the IP address in your VNC client.

Author: Chris Burton
Last update: 2019-07-30


How do I use a static network device name for KVM (to graph in Cacti for example)?

To force a static (pre-defined name for a KVM Virtual Machine you will need to edit the configuration file for the VM (replace VM_NAME with the name of the domain).

vi /etc/libvirt/qemu/VM_NAME.xml

Find the section of the file starting with "<interface type='bridge'>" and then add the following (replacing DEV_NAME with the name of the network device to use on the host).

<target dev='DEV_NAME'/>

You will then need to bring in the new change and stop/start the VM for the changes to take effect.

virsh define VM_NAME.xml
virsh shutdown VM_NAME
virsh start VM_NAME

You should now be able to see the new network device name on the host by using "ifconfig". This new device can now be picked by you Cacti for example and used to graph the VM using a static network device name.

Author: Chris Burton
Last update: 2014-12-17


How do I configure KVM to suspend/restore virtual machines when the host is rebooted?

To suspend a VM on host shutdown you will need to configure the following file

vi /etc/sysconfig/libvirt-guests

Set the following options (which are located throughout the configuration file).

ON_BOOT=start
START_DELAY=5
ON_SHUTDOWN=suspend

You might not want to use START_DELAY=5, but I would rather the virtual machines start up with a little staggering rather than concurrently.

The libvirt-guests service also needs to be enabled and started.

systemctl start libvirt-guests
systemctl enable libvirt-guests

You will now be able to reboot the host machine to install updates etc. and the virtual machines will be suspended and restored rather than shutdown interrupting their jobs.

Author: Chris Burton
Last update: 2014-12-17


How do I change the timezone on Centos 7 to UTC?

CentOS 7 now includes a tool to change the timezone, to view your current timezone run the following command.

timedatectl

This should display the current timezone configuraton. To change this to UTC (or to any other timezone) run the following.

timedatectl set-timezone UTC

You can then re-check the timezone by running the command above again.

Author: Chris Burton
Last update: 2015-01-03


How do I setup SNMPd on CentOS 7?

To setup SNMP daemon firstly install the required packages.

yum install net-snmp net-snmp-devel net-snmp-libs net-snmp-utils

Then edit the main configuration file

vi /etc/snmp/snmpd.conf

Find the line starting with "com2sec" and change the community from "public" to your secret v2c community name. If you also wish to use version v1 you can also add a line "rocommunity SECRET_COMMUNITY" at the end of the file.

You will then need to start the service

service snmpd start

And configure it to start automatically

systemctl enable snmpd

The SNMP daemon should now be running which can be used to monitor the server via Cacti etc.

 

 

Author: Chris Burton
Last update: 2015-01-14


How do I rename a KVM VM?

This example works on CentOS 7, but will likely work on other distributions too.

 

Save a copy of the Virtual Machines configuration (replacing VMNAME with the name of your VM as shown with "virsh list --all".

virsh dumpxml VMNAME > VMNAME.xml

Edit the VMNAME.xml file and search for 

<name>VMNAME</name>

and replace this with the new name for the VM and save the file.

Shutdown the VM (if the VM doesn't shutdown you can run "virsh destroy VMNAME" although data may be lost).

virsh shutdown VMNAME

To swap the server we need to undefine the old VM and define the new VM.

virsh undefine VMNAME
virsh define NEW_VMNAME VMNAME.xml

You can now start the VM to complete the rename.

virsh start NEW_VMNAME

 

Author: Chris Burton
Last update: 2015-01-29


How do I install the same RPM packages on a new server as an old?

First you need a list of the packages on the old server.

rpm -qa --qf "%{NAME}\n" > rpmlist.txt

 

Copy this file to the new server (for example using SSH).

 

To bulk install all of the packages installed on the old server

yum install `cat rpmlist.txt|tr '\n' ' '`

 

You should then be prompted to install all packages missing on the new server.

Author: Chris Burton
Last update: 2015-08-24


How do I get the disk usage of a directory including hidden directories/files?

Using du with a wildcard would not normally include "hidden" directories/files (those starting with a dot). But you can show the usage with this simple command.

du -sch .[!.]* *

 

Author: Chris Burton
Last update: 2015-08-24


How do I alter the kernel cmdline and rebuild grub2 on CentOS 7?

 

The kernel command line (cmdline) can be edited in "/etc/default/grub" (GRUB_CMDLINE_LINUX).

 

Once edited you will need to rebuild the config for grub2 using the following command

 

grub2-mkconfig --output=/boot/grub2/grub.cfg

 

You can then reboot and the new cmdline will be used (this can be verified by "cat /proc/cmdline").

 

 

Author: Chris Burton
Last update: 2016-09-15


How do I copy the partition table from disk to disk?

When using GPT you can use sgdisk to copy the partition table.

FROM="/dev/sdX"
TO="/dev/sdY"
sgdisk $FROM -R $TO
sgdisk -G $TO

This will copy the partition table over and set random GUID on the partitions on the destination device.

When using MBR you can use sfdisk to copy the partition table.

FROM="/dev/sdX"
TO="/dev/sdY"
sfdisk -d $FROM | sfdisk $TO

Author: Chris Burton
Last update: 2016-10-23


How do I show disk usage breakdown including hidden directories?

Using "du" with the following options will show usage of all files and directories including hidden (those starting with a dot).

du -sch .[!.]* *

Author: Chris Burton
Last update: 2018-08-30


How do I force apt to connect using IPv4 (or IPv6) only?

To  connect using IPv4.

apt -o Acquire::ForceIPv4=true update

Or to use IPv6 only.

apt -o Acquire::ForceIPv6=true update

Author: Chris Burton
Last update: 2019-05-06


How do I enable mDNS/zeroconf/Bonjour on Armbian?

To allow <hostname>.local domains to be resolved on Armbian install the following.

sudo apt-get install avahi-daemon libnss-mdns libnss-mymachines

You should then be able to resolve domains using mDNS.

Author: Chris Burton
Last update: 2019-05-22


How do I resize a disk image (Raspbian SD/USB image)?

To resize the last partition in a disk image, for example a Raspberry Pi SD card image first increase the size of the file on disk using truncate, here we're going to increase it by 500MB.

truncate 2019-06-20-raspbian-buster-full.img --size=+500M

Once the file size has been increased the size of the second partition need to be updated to use all of the remaining space.

parted --script ./2019-06-20-raspbian-buster-full.img resizepart 2 100%

And then the filesystem will need to be extended to the size of the new partition, first enable access to the partitions in the disk image.

losetup -fP --show 2019-06-20-raspbian-buster-full.img

This will return the loop device name, in this example we're going to assume it's "/dev/loop1". 

Then check the filesystem.

e2fsck -f /dev/loop1p2

Now we can do the actual filesystem resize.

resize2fs /dev/loop1p2

If you see no errors then we're done and we can remove the loop interface.

losetup -d /dev/loop1

The size of the disk image has now been extended and the second partition has been increased to use all of the space.

Author: Chris Burton
Last update: 2019-07-24


ClusterCTRL: How do I boot a Compute Module from eMMC?

To boot a Compute Module from the eMMC USBBOOT must be disabled.

If your Compute Module doesn't already have a bootable system see our other instructions.

Background

When using ClusterCTRL with Compute Modules "clusterctrl status" will show the status for both Power "pX:S" (X=Node number, S=Status 0=on/1=off) and USBBOOT "uX:S" (X=Node number, S=Status on/off).

uX:0 Try to boot from the onboard eMMC and fallback to booting as a USB device.
uX:1 Boot as a USB device.

You will also see a "ctrl_bus:" line, this shows the details for all ClusterCTRL device found (space separate if you have multiple ClusterCTRL devices).

For each ClusterCTRL device you will see 3 numbers, for example.

ctrl_bus:20:3:3

The first number above "20" shows the ORDER, this is used if you have multiple ClusterCTRL devices on the same controller (lower numbers have a higher priority). A ClusterHAT always has highest priority so if present will be used for p1-p4. It will then assign the next pX.. numbers to the ClusterCTRL device with the lowest ORDER, then the next lowest etc (you may need this ORDER value to save the state below).

The second number "3" is the I2C bus (/dev/i2c-3 for example) used to communicate with the ClusterCTRL device.

The last number "3" is the maximum number of nodes this device supports.

Disable USBBOOT

Power of the node and disable USBBOOT.

clusterctrl off pX
clusterctrl usbboot off pX

The next time the node is powered on it will boot from onboard eMMC.

Save power on state (optional)

If you want the node to always boot from eMMC you will need to save the state (replacing ORDER with the order number shown in the "clusterctrl status" output above).

clusterctrl save ORDER

This command saves both power and USBBOOT states for use on next power up

Power on

You can now power on the node and it will boot from the eMMC.

clusterctrl on pX

Within a few seconds you should see it boot up in the logs, the first boot will be slower as it reconfigures and resizes the filesystem (CTRL-c to exit).

tail -f /var/log/kern.log /var/log/daemon.log

You can now accesses the node as normal "ping pX.local", "ssh pi@pX.local", if you haven't enabled SSH you can access it using the serial console "screen /dev/ttypiX" (CTRL-x then 'k' to exit).

Author: Chris Burton
Last update: 2019-08-09


ClusterCTRL: How do I USBBOOT a Compute Module?

To boot a Compute Module as a USB device USBBOOT must be enabled.

Background

When using ClusterCTRL with Compute Modules "clusterctrl status" will show the status for both Power "pX:S" (X=Node number, S=Status 0=on/1=off) and USBBOOT "uX:S" (X=Node number, S=Status on/off).

uX:0 Try to boot from the onboard eMMC and fallback to booting as a USB device.
uX:1 Boot as a USB device.

You will also see a "ctrl_bus:" line, this shows the details for all ClusterCTRL device found (space separate if you have multiple ClusterCTRL devices).

For each ClusterCTRL device you will see 3 numbers, for example.

ctrl_bus:20:3:3

The first number above "20" shows the ORDER, this is used if you have multiple ClusterCTRL devices on the same controller (lower numbers have a higher priority). A ClusterHAT always has highest priority so if present will be used for p1-p4. It will then assign the next pX.. numbers to the ClusterCTRL device with the lowest ORDER, then the next lowest etc (you may need this ORDER value to save the state below).

The second number "3" is the I2C bus (/dev/i2c-3 for example) used to communicate with the ClusterCTRL device.

The last number "3" is the maximum number of nodes this device supports.

Enable USBBOOT

Power of the node and disable USBBOOT.

clusterctrl off pX
clusterctrl usbboot on pX

The next time the node is powered on it will boot as a USB device.

Save power on state (optional)

If you want the node to always boot using USBBOOT you will need to save the state (replacing ORDER with the order number shown in the "clusterctrl status" output above).

clusterctrl save ORDER

This command saves both power and USBBOOT states for use on next power up

You can now follow our standard instructions on using USBBOOT.

Author: Chris Burton
Last update: 2019-08-06


ClusterCTRL: How do I USBBOOT a Compute Module and use eMMC as local storage/swap?

A Compute Module will boot as a USB device if either USBBOOT is enabled or it can't find bootcode.txt in the first FAT partition on the eMMC.

Replace X in the following steps with X of the pX device you're configuring (for X=1 for p1, X=5 for p5, etc).

First setup the filesystem and Compute Module to boot as a USB Device.

Once USBBOOT is working power on the Compute Module "clusterctrl on pX" and wait for it to boot - (until you can "ping pX.local", etc).

On the controller you will now need to disable USBBOOT.

clusterctrl usbboot off pX

To permanently allow access to the eMMC as local storage you will need to save the power on state (see the USBBOOT instructions on how to do this).

Log into the Compute Module node (via either "ssh pi@pX.local" or "screen /dev/ttypiX").

You can now access the eMMC as /dev/mmcblk0 which can be partitioned with fdisk/etc.

The existing partitions can be viewed with fdisk

sudo fdisk -l /dev/mmcblk0

To ensure the Compute Module boots as a USB Device (and not from the eMMC) you will need to ensure the first FAT partition does not contain bootcode.bin file (either by removing the file or not having a FAT partition at all).

For example to use it as a 1GB swap partition and the rest as a /data partition you could do the following.

Using fdisk remove the existing partitions (this will remove all existing data so backup first).

sudo fdisk /dev/mmcblk0

Press "p" and return to see the exising partitions.

For each existing partition press "d" and return, enter the partition number or press enter to delete the last one each time.

Once the existing partitions are removed you can add the new partitions.

Type "n" and return, use "p" for a primary partition, use parition number 1, it will start with the first available sector so you can press enter, for the size enter "+1G". To set the type to "Linux Swap" press "t" return, then type "82" (Linux Swap) and press return.

Then add the data partition, again type "n" and return, use "p" for a primary partition, use parition number 2, go with the default first and end sectors by pressing enter twice.

You can then use "p" and enter to see the partitions.

If you're happy use "w" and return to write the changes and exit or "q" to quit without changing the partition table.

Reload the partition table

sudo partprobe

Setup swap and create the filesystem

sudo mkswap /dev/mmcblk0p1

This will show you the UUID for the swap partition, keep this for later.

sudo mke2fs -j /dev/mmcblk0p2

Again this will show the UUID you will need later.

Create the mount point for the filesystem, here I'm using /data

sudo mkdir /data

Edit /etc/fstab to setup using both swap and the new filesystem

sudo nano /etc/fstab

Append new lines for both swap and data partitions (replacing with your UUID from above).

UUID=029577ad-b1d2-43ad-b1f9-348e37969049       swap    swap defaults 0 0
UUID=e5dd1581-8baf-43d7-b49b-a24cb93da970       /data   ext4 defaults 0 0

Write the file and exit nano.

Check the swap partition is setup OK in /etc/fstab

sudo swapon -a

You should now see the added swap space if you run "free".

Mount the filesystem.

sudo mount -a

You should see the mounted filesystem by running "df -h".

After rebooting the Compute Module it will still boot via USBBOOT and use the new local swap and /data filesystem.

Author: Chris Burton
Last update: 2019-08-06


What does the output of 'clusterctrl status' mean?

Running "clusterctrl status" you will get output similar to the below text.

clusterhat:1
clusterctrl:2
maxpi:10
ctrl_bus:20:4:3 21:3:3
hat_version:1.2
hat_version_major:1
hat_version_minor:2
hat_size:4
hat_uuid:622aef87-17d6-4022-a370-781e1c3fe2b9
hat_vendor:8086 Consultancy
hat_product_id:0x0004
throttled:0x0
hat_alert:0
p1:0
p2:0
p3:0
p4:0
ctrl20:FW:1.1 ADC1:5014mV T1:18.60C
p5:0
u5:1
p6:0
u6:1
p7:0
u7:1
ctrl21:FW:1.1 ADC1:4891mV T1:23.52C
p8:0
u8:1
p9:0
u9:1
p10:0
u10:1

clusterhat:
False if no Cluster HAT is detected.
1 if a Cluster HAT was detected.

clusterctrl:
False if no Cluster CTRL devices (Single/Triple/Stack/A+6) detected
X number of devices detected

maxpi:
A count of the number of nodes controllable.

ctrl_bus:
Space separated list of data (order:I2C device:maxpi) from each Cluster CTRL device.

From the example above
20:4:3 - Cluster CTRL device with order 20, is controllable via /dev/i2c-4 and controls 3 nodes.
21:3:3 - Cluster CTRL device with order 21, is controllable via /dev/i2c-3 and controls 3 nodes.

hat_version:
Full version number of the Cluster HAT detected (1.2, 2.3, etc).

hat_version_major:
Major Cluster HAT version (1 for 1.x, 2 for 2.x, etc).

hat_version_minor:
Minor part of the version number (4 for 1.4, 3 for 2.3, etc).

hat_size:
Number of Pi Zeros in the Cluster HAT this is by default 4, the maximum number supported. It can be overriden by editing /etc/defaults/clusterctrl and changing the value for CLUSTERHAT_SIZE. This will change the number of nodes controlled by the Cluster HAT for example if you only have 3 Pi Zero you can use CLUSTERHAT_SIZE=3 and the p4 name would be used for Cluster CTRL nodes.

hat_uuid:
The UUID for the Cluster HAT device, unique for every product.

hat_vendor:
Vendor name "8086 Consultancy".

hat_product_id:
8086 Consultancy product id for the HAT.

throttled:
Output of "vcgencmd get_throttled" shows any power/thermal issues on the controller.

hat_alert:
Status of the ALERT LED (0=off, 1=on).

pX:
Status of node pX power (0=off, 1=on).

uX: [only shown on Cluster CTRL devices]
Status of usbboot for node pX (0=off, 1=on).

ctrlX: [only shown for Cluster CTRL devices]
Space separated fields for FW: firmware version and where supported ADCx: voltages, (inaccurate) Tx: temperature, etc.

Author: Chris Burton
Last update: 2020-11-29


How are nodes numbered with Cluster CTRL/Cluster HAT (resolve order conflict)?

The pX number of a node for control is set by it's priority (this might not be the same as the hostname/configuration of the image/usbboot filesystem).

Any detected Cluster HAT (v1.x or v2.x) is given highest priority so this will always be p1-p4 if it's detected.

All Cluster CTRL devices (Triple/A+6/etc.) have an "order" stored in EEPROM/FLASH. By default the order is set to 20 for the Triple and 10 for the A+6. The lower this "order" number the higher the priority.

So if you have ClusterCTRL Triple devices with order 20 and 21 the node numbers will be as follows.

With Cluster HAT
p1-p4 ClusterHAT
p5-p7 Triple (priority 20)
p8-p10 Triple (priority 21)

Without Cluster HAT
p1-p3 Triple (priority 20)
p4-p6 Triple (priority 21)

If you have more than one ClusterCTRL device with the same "order" the "clusterctrl" tool will show an error, you would need to remove all but one of the conflicting ClusterCTRL devices to set a new order.

$ clusterctrl status
ERROR: Duplicate ClusterCTRL 'order' found
I2C Bus: 4 Order: 20
I2C Bus: 3 Order: 20

Changing ClusterCTRL devices order

If you have a conflict first unplug all but one device with the same order.

To  change the order stored of a ClusterCTRL device you need the existing order number which can be seen using "clusterctrl status".

To temporarily change the order of a Cluster CTRL device use "clusterctrl setorder <old order> <new order>", so to change from order 20 to order 21 run.

clusterctrl setorder 20 21

The order can be set between 1 and 255.

The order is now set temporarily and will revert back to the old order after a power cycle or reset. To make the change perminant the settings must be saved to EEPROM.

clusterctrl save 21

The new order (and usbboot/power) will now be saved to the EEPROM and used at next power on.

Author: Chris Burton
Last update: 2019-08-08


How do I enable autologin on TTY running on the Controllers serial port [Raspbian Buster/Bullseye/Bookworm]?

To enable auto logins on the TTY running on any serial port on the Controller log in and run the following commands changing "USERNAME" to your username and reboot.

sudo mkdir -p /etc/systemd/system/serial-getty@ttyS0.service.d/
sudo bash -c "cat > /etc/systemd/system/serial-getty@ttyS0.service.d/autologin.conf << EOF
[Service]
ExecStart=
ExecStart=-/sbin/agetty --autologin USERNAME --noclear %I \$TERM
EOF"

And to disable it run the following command and reboot.

sudo rm -rf /etc/systemd/system/serial-getty@ttyS0.service.d/

When enabling this all users with access to the Controller Pi serial port will be able to log in without any authentication.

Author: Chris Burton
Last update: 2024-01-02


How do I stop the HAT EEPROM being loaded on a Raspberry Pi?

To prevent the device tree configuration from being loaded from a Raspberry Pi HAT add the following to your config.txt file.

force_eeprom_read=0

Author: Chris Burton
Last update: 2019-11-16


How do I manually install clusterctrl?

If you're using one of the provided ClusterCTRL images the cluserctrl tool will already be installed but those using the ClusterCTRL Stack or only want to manage power for a ClusterHAT for example may want to install it manually.

To install the tool manually you can normally run through the following steps as the root user (you can usually switch to the root user by using "sudo -i").

TMPINSTALL=/tmp/clusterhat
apt update
apt install git libusb-1.0-0-dev python3-usb python3-libusb1 python3-smbus python-is-python3 # When using Python3
# apt install git libusb-1.0-0-dev python-usb python-libusb1 python-smbus # When using older installs with Python2
git clone https://github.com/burtyb/clusterhat-image.git $TMPINSTALL
mkdir /usr/share/clusterctrl/
cp $TMPINSTALL/files/usr/sbin/cluster* /usr/sbin/
cp $TMPINSTALL/files/usr/share/clusterctrl/default-clusterctrl /etc/default/clusterctrl
cp $TMPINSTALL/files/etc/udev/rules.d/90-clusterctrl.rules /etc/udev/rules.d/
cp $TMPINSTALL/files/usr/lib/systemd/system/clusterctrl-init.service /usr/lib/systemd/system/
cp -r $TMPINSTALL/files/usr/share/clusterctrl/python/ /usr/share/clusterctrl/
echo 'TYPE=c' >> /etc/default/clusterctrl
udevadm control --reload-rules
systemctl enable clusterctrl-init
raspi-config nonint do_i2c 0 # Enable I2C
rm -rf $TMPINSTALL

If you see an error on any of these stages please create a new thread on the support forum or open a support ticket with details of the distribution you're using and the issue you're having.

Now disconnect and reconnect the USB cable (if using a ClusterCTRL device) and then check it's found the device.

clusterctrl status

This should then find and show the status of either a ClusterHAT and/or any ClusterCTRL devices you have connected.

Author: Chris Burton
Last update: 2023-11-26


How do I disable automount in Raspberry Pi OS (Bullseye with GUI)?

Create a file ~/.config/pcmanfm/LXDE-pi/pcmanfm.conf for the "default" user (the one created when installed) with the following contents.

[volume]
mount_on_startup=0
mount_removable=0
autorun=1

After a reboot USB disks/etc. will not be automatically mounted anymore.

Author: Chris Burton
Last update: 2023-07-16


Software » Linux » Manuals

Manual: crontab

 

NAME

crontab - maintain crontab files for individual users (ISC Cron V4.1)  

SYNOPSIS

crontab [-u user] file
crontab [-u user] [-l | -r | -e] [-i] [-s]  

DESCRIPTION

Crontab is the program used to install, deinstall or list the tables used to drive the cron(8) daemon in ISC Cron. Each user can have their own crontab, and though these are files in /var/spool/ , they are not intended to be edited directly. For SELinux in mls mode can be ev en more crontabs - for each range. For more see selinux(8).

If the cron.allow file exists, then you must be listed therein in order to be allowed to use this command. If the cron.allow file does not exist but the cron.deny file does exist, then you must not be listed in the cron.deny file in order to use this command. If neither of these files exists, only the super user will be allowed to use this command.

 

OPTIONS

-u
It specifies the name of the user whose crontab is to be tweaked. If this optio n is not given, crontab examines "your" crontab, i.e., the crontab of the person executing the command. Note that su(8) can confuse crontab and that if you are running inside of su(8) you should always use the -u option for safety's sake. The first form of this command is used to install a new crontab from some named file or standard input if the pseudo-filename "-" is given.
-l
The current crontab will be displayed on standard output.
-r
The current crontab will be be removed.
-e
This option is used to edit the current crontab using the editor specified by the VISUAL or EDITOR environment v ariables. After you exit from the editor, the modified crontab will be installed automatically.
-i
This option modifies the -r option to prompt the user for a 'y/Y' response before actually removing the crontab.
-s
It will append the current SELinux security context string as an MLS_LEVEL setting to the crontab file before editing / replacement occurs - see the documentation of MLS_LEVEL in crontab(5).

 

SEE ALSO

crontab(5), cron(8)  

FILES

/etc/cron.allow
/etc/cron.deny

 

STANDARDS

The crontab command conforms to IEEE Std1003.2-1992 (``POSIX''). This new command syntax differs from previous versions of Vixie Cron, as well as from the classic SVR3 syntax.  

DIAGNOSTICS

A fairly informative usage message appears if you run it with a bad command line.  

AUTHOR

Paul Vixie <vixie@isc.org>

 

Index

NAME
SYNOPSIS
DESCRIPTION
OPTIONS
SEE ALSO
FILES
STANDARDS
DIAGNOSTICS
AUTHOR

Author: Chris Burton
Last update: 2014-12-16


Manual: vi

PROLOG

This manual page is part of the POSIX Programmer's Manual. The Linux implementation of this interface may differ (consult the corresponding Linux manual page for details of Linux behavior), or the interface may not be implemented on Linux.  

NAME

vi - screen-oriented (visual) display editor  

SYNOPSIS

vi [-rR][-c command][-t tagstring][-w size][file ...]   

DESCRIPTION

This utility shall be provided on systems that both support the User Portability Utilities option and define the POSIX2_CHAR_TERM symbol. On other systems it is optional.

The vi (visual) utility is a screen-oriented text editor. Only the open and visual modes of the editor are described in IEEE Std 1003.1-2001; see the line editor ex for additional editing capabilities used in vi. The user can switch back and forth between vi and ex and execute ex commands from within vi.

This reference page uses the term edit buffer to describe the current working text. No specific implementation is implied by this term. All editing changes are performed on the edit buffer, and no changes to it shall affect any file until an editor command writes the file.

When using vi, the terminal screen acts as a window into the editing buffer. Changes made to the editing buffer shall be reflected in the screen display; the position of the cursor on the screen shall indicate the position within the editing buffer.

Certain terminals do not have all the capabilities necessary to support the complete vi definition. When these commands cannot be supported on such terminals, this condition shall not produce an error message such as "not an editor command" or report a syntax error. The implementation may either accept the commands and produce results on the screen that are the result of an unsuccessful attempt to meet the requirements of this volume of IEEE Std 1003.1-2001 or report an error describing the terminal-related deficiency.  

OPTIONS

The vi utility shall conform to the Base Definitions volume of IEEE Std 1003.1-2001, Section 12.2, Utility Syntax Guidelines.

The following options shall be supported:

-c  command
See the ex command description of the -c option.
-r
See the ex command description of the -r option.
-R
See the ex command description of the -R option.
-t  tagstring
See the ex command description of the -t option.
-w  size
See the ex command description of the -w option.

 

 

OPERANDS

See the OPERANDS section of the ex command for a description of the operands supported by the vi command.  

STDIN

If standard input is not a terminal device, the results are undefined. The standard input consists of a series of commands and input text, as described in the EXTENDED DESCRIPTION section.

If a read from the standard input returns an error, or if the editor detects an end-of-file condition from the standard input, it shall be equivalent to a SIGHUP asynchronous event.  

INPUT FILES

See the INPUT FILES section of the ex command for a description of the input files supported by the vi command.  

ENVIRONMENT VARIABLES

See the ENVIRONMENT VARIABLES section of the ex command for the environment variables that affect the execution of the vi command.  

ASYNCHRONOUS EVENTS

See the ASYNCHRONOUS EVENTS section of the ex for the asynchronous events that affect the execution of the vi command.  

STDOUT

If standard output is not a terminal device, undefined results occur.

Standard output may be used for writing prompts to the user, for informational messages, and for writing lines from the file.  

STDERR

If standard output is not a terminal device, undefined results occur.

The standard error shall be used only for diagnostic messages.  

OUTPUT FILES

See the OUTPUT FILES section of the ex command for a description of the output files supported by the vi command.  

EXTENDED DESCRIPTION

If the terminal does not have the capabilities necessary to support an unspecified portion of the vi definition, implementations shall start initially in ex mode or open mode. Otherwise, after initialization, vi shall be in command mode; text input mode can be entered by one of several commands used to insert or change text. In text input mode, <ESC> can be used to return to command mode; other uses of <ESC> are described later in this section; see Terminate Command or Input Mode .  

Initialization in ex and vi

See Initialization in ex and vi for a description of ex and vi initialization for the vi utility.  

Command Descriptions in vi

The following symbols are used in this reference page to represent arguments to commands.

buffer
See the description of buffer in the EXTENDED DESCRIPTION section of the ex utility; see Command Descriptions in ex .

In open and visual mode, when a command synopsis shows both [ buffer] and [ count] preceding the command name, they can be specified in either order.

count
A positive integer used as an optional argument to most commands, either to give a repeat count or as a size. This argument is optional and shall default to 1 unless otherwise specified.

The Synopsis lines for the vi commands <control>-G, <control>-L, <control>-R, <control>-], %, &, ^, D, m, M, Q, u, U, and ZZ do not have count as an optional argument. Regardless, it shall not be an error to specify a count to these commands, and any specified count shall be ignored.

motion
An optional trailing argument used by the !, <, >, c, d, and y commands, which is used to indicate the region of text that shall be affected by the command. The motion can be either one of the command characters repeated or one of several other vi commands (listed in the following table). Each of the applicable commands specifies the region of text matched by repeating the command; each command that can be used as a motion command specifies the region of text it affects.

Commands that take motion arguments operate on either lines or characters, depending on the circumstances. When operating on lines, all lines that fall partially or wholly within the text region specified for the command shall be affected. When operating on characters, only the exact characters in the specified text region shall be affected. Each motion command specifies this individually.

When commands that may be motion commands are not used as motion commands, they shall set the current position to the current line and column as specified.

The following commands shall be valid cursor motion commands:

 

<apostrophe>       (    -    j    H
<carriage-return>  )    $    k    L
<comma>            [[   %    l    M
<control>-H        ]]   _    n    N
<control>-N        {    ;    t    T
<control>-P        }    ?    w    W
<grave accent>     ^    b    B
<newline>          +    e    E
<space>            |    f    F
<zero>             /    h    G

Any count that is specified to a command that has an associated motion command shall be applied to the motion command. If a count is applied to both the command and its associated motion command, the effect shall be multiplicative.

 

The following symbols are used in this section to specify locations in the edit buffer:

current character

The character that is currently indicated by the cursor.

end of a line

The point located between the last non- <newline> (if any) and the terminating <newline> of a line. For an empty line, this location coincides with the beginning of the line.

end of the edit buffer

The location corresponding to the end of the last line in the edit buffer.

 

The following symbols are used in this section to specify command actions:

bigword
In the POSIX locale, vi shall recognize four kinds of bigwords:
1.
A maximal sequence of non- <blank>s preceded and followed by <blank>s or the beginning or end of a line or the edit buffer

 

2.
One or more sequential blank lines

 

3.
The first character in the edit buffer

 

4.
The last non- <newline> in the edit buffer

 

word
In the POSIX locale, vi shall recognize five kinds of words:
1.
A maximal sequence of letters, digits, and underscores, delimited at both ends by:
*
Characters other than letters, digits, or underscores

 

*
The beginning or end of a line

 

*
The beginning or end of the edit buffer

 

 

2.
A maximal sequence of characters other than letters, digits, underscores, or <blank>s, delimited at both ends by:
*
A letter, digit, underscore

 

*
<blank>s

 

*
The beginning or end of a line

 

*
The beginning or end of the edit buffer

 

 

3.
One or more sequential blank lines

 

4.
The first character in the edit buffer

 

5.
The last non- <newline> in the edit buffer

 

section boundary

A section boundary is one of the following:

1.
A line whose first character is a <form-feed>

 

2.
A line whose first character is an open curly brace ( '{' )

 

3.
A line whose first character is a period and whose second and third characters match a two-character pair in the sections edit option (see ed)

 

4.
A line whose first character is a period and whose only other character matches the first character of a two-character pair in the sections edit option, where the second character of the two-character pair is a <space>

 

5.
The first line of the edit buffer

 

6.
The last line of the edit buffer if the last line of the edit buffer is empty or if it is a ]] or } command; otherwise, the last non- <newline> of the last line of the edit buffer

 

paragraph boundary

A paragraph boundary is one of the following:

1.
A section boundary

 

2.
A line whose first character is a period and whose second and third characters match a two-character pair in the paragraphs edit option (see ed)

 

3.
A line whose first character is a period and whose only other character matches the first character of a two-character pair in the paragraphs edit option, where the second character of the two-character pair is a <space>

 

4.
One or more sequential blank lines

 

remembered search direction

See the description of remembered search direction in ed.

sentence boundary

A sentence boundary is one of the following:

1.
A paragraph boundary

 

2.
The first non- <blank> that occurs after a paragraph boundary

 

3.
The first non- <blank> that occurs after a period ( '.' ), exclamation mark ( '!' ), or question mark ( '?' ), followed by two <space>s or the end of a line; any number of closing parenthesis ( ')' ), closing brackets ( ']' ), double quote ( ' ),' or single quote ( '" ) characters can appear between the punctuation mark and the two <space>s or end-of-line

 

 

In the remainder of the description of the vi utility, the term "buffer line" refers to a line in the edit buffer and the term "display line" refers to the line or lines on the display screen used to display one buffer line. The term "current line" refers to a specific "buffer line".

If there are display lines on the screen for which there are no corresponding buffer lines because they correspond to lines that would be after the end of the file, they shall be displayed as a single tilde ( '~' ) character, plus the terminating <newline>.

The last line of the screen shall be used to report errors or display informational messages. It shall also be used to display the input for "line-oriented commands" ( /, ?, :, and !). When a line-oriented command is executed, the editor shall enter text input mode on the last line on the screen, using the respective command characters as prompt characters. (In the case of the ! command, the associated motion shall be entered by the user before the editor enters text input mode.) The line entered by the user shall be terminated by a <newline>, a non- <control>-V-escaped <carriage-return>, or unescaped <ESC>. It is unspecified if more characters than require a display width minus one column number of screen columns can be entered.

If any command is executed that overwrites a portion of the screen other than the last line of the screen (for example, the ex suspend or ! commands), other than the ex shell command, the user shall be prompted for a character before the screen is refreshed and the edit session continued.

<tab>s shall take up the number of columns on the screen set by the tabstop edit option (see ed), unless there are less than that number of columns before the display margin that will cause the displayed line to be folded; in this case, they shall only take up the number of columns up to that boundary.

The cursor shall be placed on the current line and relative to the current column as specified by each command described in the following sections.

In open mode, if the current line is not already displayed, then it shall be displayed.

In visual mode, if the current line is not displayed, then the lines that are displayed shall be expanded, scrolled, or redrawn to cause an unspecified portion of the current line to be displayed. If the screen is redrawn, no more than the number of display lines specified by the value of the window edit option shall be displayed (unless the current line cannot be completely displayed in the number of display lines specified by the window edit option) and the current line shall be positioned as close to the center of the displayed lines as possible (within the constraints imposed by the distance of the line from the beginning or end of the edit buffer). If the current line is before the first line in the display and the screen is scrolled, an unspecified portion of the current line shall be placed on the first line of the display. If the current line is after the last line in the display and the screen is scrolled, an unspecified portion of the current line shall be placed on the last line of the display.

In visual mode, if a line from the edit buffer (other than the current line) does not entirely fit into the lines at the bottom of the display that are available for its presentation, the editor may choose not to display any portion of the line. The lines of the display that do not contain text from the edit buffer for this reason shall each consist of a single '@' character.

In visual mode, the editor may choose for unspecified reasons to not update lines in the display to correspond to the underlying edit buffer text. The lines of the display that do not correctly correspond to text from the edit buffer for this reason shall consist of a single '@' character (plus the terminating <newline>), and the <control>-R command shall cause the editor to update the screen to correctly represent the edit buffer.

Open and visual mode commands that set the current column set it to a column position in the display, and not a character position in the line. In this case, however, the column position in the display shall be calculated for an infinite width display; for example, the column related to a character that is part of a line that has been folded onto additional screen lines will be offset from the display line column where the buffer line begins, not from the beginning of a particular display line.

The display cursor column in the display is based on the value of the current column, as follows, with each rule applied in turn:

1.
If the current column is after the last display line column used by the displayed line, the display cursor column shall be set to the last display line column occupied by the last non- <newline> in the current line; otherwise, the display cursor column shall be set to the current column.

 

2.
If the character of which some portion is displayed in the display line column specified by the display cursor column requires more than a single display line column:
a.
If in text input mode, the display cursor column shall be adjusted to the first display line column in which any portion of that character is displayed.

 

b.
Otherwise, the display cursor column shall be adjusted to the last display line column in which any portion of that character is displayed.

 

 

The current column shall not be changed by these adjustments to the display cursor column.

If an error occurs during the parsing or execution of a vi command:

*
The terminal shall be alerted. Execution of the vi command shall stop, and the cursor (for example, the current line and column) shall not be further modified.

 

*
Unless otherwise specified by the following command sections, it is unspecified whether an informational message shall be displayed.

 

*
Any partially entered vi command shall be discarded.

 

*
If the vi command resulted from a map expansion, all characters from that map expansion shall be discarded, except as otherwise specified by the map command (see ed).

 

*
If the vi command resulted from the execution of a buffer, no further commands caused by the execution of the buffer shall be executed.

 

Page Backwards

Synopsis:

 

[count] <control>-B

 

If in open mode, the <control>-B command shall behave identically to the z command. Otherwise, if the current line is the first line of the edit buffer, it shall be an error.

If the window edit option is less than 3, display a screen where the last line of the display shall be some portion of:

 

(current first line) -1

otherwise, display a screen where the first line of the display shall be some portion of:

 

(current first line) - count x ((window edit option) -2)

If this calculation would result in a line that is before the first line of the edit buffer, the first line of the display shall display some portion of the first line of the edit buffer.

Current line: If no lines from the previous display remain on the screen, set to the last line of the display; otherwise, set to ( line - the number of new lines displayed on this screen).

Current column: Set to non- <blank>.  

Scroll Forward

Synopsis:

 

[count] <control>-D

 

If the current line is the last line of the edit buffer, it shall be an error.

If no count is specified, count shall default to the count associated with the previous <control>-D or <control>-U command. If there was no previous <control>-D or <control>-U command, count shall default to the value of the scroll edit option.

If in open mode, write lines starting with the line after the current line, until count lines or the last line of the file have been written.

Current line: If the current line + count is past the last line of the edit buffer, set to the last line of the edit buffer; otherwise, set to the current line + count.

Current column: Set to non- <blank>.  

Scroll Forward by Line

Synopsis:

 

[count] <control>-E

 

Display the line count lines after the last line currently displayed.

If the last line of the edit buffer is displayed, it shall be an error. If there is no line count lines after the last line currently displayed, the last line of the display shall display some portion of the last line of the edit buffer.

Current line: Unchanged if the previous current character is displayed; otherwise, set to the first line displayed.

Current column: Unchanged.  

Page Forward

Synopsis:

 

[count] <control>-F

 

If in open mode, the <control>-F command shall behave identically to the z command. Otherwise, if the current line is the last line of the edit buffer, it shall be an error.

If the window edit option is less than 3, display a screen where the first line of the display shall be some portion of:

 

(current last line) +1

otherwise, display a screen where the first line of the display shall be some portion of:

 

(current first line) + count x ((window edit option) -2)

If this calculation would result in a line that is after the last line of the edit buffer, the last line of the display shall display some portion of the last line of the edit buffer.

Current line: If no lines from the previous display remain on the screen, set to the first line of the display; otherwise, set to ( line + the number of new lines displayed on this screen).

Current column: Set to non- <blank>.  

Display Information

Synopsis:

 

<control>-G

 

This command shall be equivalent to the ex file command.  

Move Cursor Backwards

Synopsis:

 

[count] <control>-H

[count] h

the current erase character (see stty)

 

If there are no characters before the current character on the current line, it shall be an error. If there are less than count previous characters on the current line, count shall be adjusted to the number of previous characters on the line.

If used as a motion command:

1.
The text region shall be from the character before the starting cursor up to and including the countth character before the starting cursor.

 

2.
Any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Unchanged.

Current column: Set to ( column - the number of columns occupied by count characters ending with the previous current column).  

Move Down

Synopsis:

 

[count] <newline>

[count] <control>-J

[count] <control>-M

[count] <control>-N

[count] j

[count] <carriage-return>

[count] +

 

If there are less than count lines after the current line in the edit buffer, it shall be an error.

If used as a motion command:

1.
The text region shall include the starting line and the next count - 1 lines.

 

2.
Any text copied to a buffer shall be in line mode.

 

If not used as a motion command:

Current line: Set to current line+ count.

Current column: Set to non- <blank> for the <carriage-return>, <control>-M, and + commands; otherwise, unchanged.  

Clear and Redisplay

Synopsis:

 

<control>-L

 

If in open mode, clear the screen and redisplay the current line. Otherwise, clear and redisplay the screen.

Current line: Unchanged.

Current column: Unchanged.  

Move Up

Synopsis:

 

[count] <control>-P

[count] k

[count] -

 

If there are less than count lines before the current line in the edit buffer, it shall be an error.

If used as a motion command:

1.
The text region shall include the starting line and the previous count lines.

 

2.
Any text copied to a buffer shall be in line mode.

 

If not used as a motion command:

Current line: Set to current line - count.

Current column: Set to non- <blank> for the - command; otherwise, unchanged.  

Redraw Screen

Synopsis:

 

<control>-R

 

If any lines have been deleted from the display screen and flagged as deleted on the terminal using the @ convention (see the beginning of the EXTENDED DESCRIPTION section), they shall be redisplayed to match the contents of the edit buffer.

It is unspecified whether lines flagged with @ because they do not fit on the terminal display shall be affected.

Current line: Unchanged.

Current column: Unchanged.  

Scroll Backward

Synopsis:

 

[count] <control>-U

 

If the current line is the first line of the edit buffer, it shall be an error.

If no count is specified, count shall default to the count associated with the previous <control>-D or <control>-U command. If there was no previous <control>-D or <control>-U command, count shall default to the value of the scroll edit option.

Current line: If count is greater than the current line, set to 1; otherwise, set to the current line - count.

Current column: Set to non- <blank>.  

Scroll Backward by Line

Synopsis:

 

[count] <control>-Y

 

Display the line count lines before the first line currently displayed.

If the current line is the first line of the edit buffer, it shall be an error. If this calculation would result in a line that is before the first line of the edit buffer, the first line of the display shall display some portion of the first line of the edit buffer.

Current line: Unchanged if the previous current character is displayed; otherwise, set to the first line displayed.

Current column: Unchanged.  

Edit the Alternate File

Synopsis:

 

<control>-^

This command shall be equivalent to the ex edit command, with the alternate pathname as its argument.

 

Terminate Command or Input Mode

Synopsis:

 

<ESC>

 

If a partial vi command (as defined by at least one, non- count character) has been entered, discard the count and the command character(s).

Otherwise, if no command characters have been entered, and the <ESC> was the result of a map expansion, the terminal shall be alerted and the <ESC> character shall be discarded, but it shall not be an error.

Otherwise, it shall be an error.

Current line: Unchanged.

Current column: Unchanged.  

Search for tagstring

Synopsis:

 

<control>-]

 

If the current character is not a word or <blank>, it shall be an error.

This command shall be equivalent to the ex tag command, with the argument to that command defined as follows.

If the current character is a <blank>:

1.
Skip all <blank>s after the cursor up to the end of the line.

 

2.
If the end of the line is reached, it shall be an error.

 

Then, the argument to the ex tag command shall be the current character and all subsequent characters, up to the first non-word character or the end of the line.  

Move Cursor Forward

Synopsis:

 

[count] <space>

[count] l  (ell)

 

If there are less than count non- <newline>s after the cursor on the current line, count shall be adjusted to the number of non- <newline>s after the cursor on the line.

If used as a motion command:

1.
If the current or countth character after the cursor is the last non- <newline> in the line, the text region shall be comprised of the current character up to and including the last non- <newline> in the line. Otherwise, the text region shall be from the current character up to, but not including, the countth character after the cursor.

 

2.
Any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

If there are no non- <newline>s after the current character on the current line, it shall be an error.

Current line: Unchanged.

Current column: Set to the last column that displays any portion of the countth character after the current character.  

Replace Text with Results from Shell Command

Synopsis:

 

[count] ! motion shell-commands <newline>

 

If the motion command is the ! command repeated:

1.
If the edit buffer is empty and no count was supplied, the command shall be the equivalent of the ex :read ! command, with the text input, and no text shall be copied to any buffer.

 

2.
Otherwise:
a.
If there are less than count -1 lines after the current line in the edit buffer, it shall be an error.

 

b.
The text region shall be from the current line up to and including the next count -1 lines.

 

 

Otherwise, the text region shall be the lines in which any character of the text region specified by the motion command appear.

Any text copied to a buffer shall be in line mode.

This command shall be equivalent to the ex ! command for the specified lines.  

Move Cursor to End-of-Line

Synopsis:

 

[count] $

 

It shall be an error if there are less than ( count -1) lines after the current line in the edit buffer.

If used as a motion command:

1.
If count is 1:
a.
It shall be an error if the line is empty.

 

b.
Otherwise, the text region shall consist of all characters from the starting cursor to the last non- <newline> in the line, inclusive, and any text copied to a buffer shall be in character mode.

 

 

2.
Otherwise, if the starting cursor position is at or before the first non- <blank> in the line, the text region shall consist of the current and the next count -1 lines, and any text saved to a buffer shall be in line mode.

 

3.
Otherwise, the text region shall consist of all characters from the starting cursor to the last non- <newline> in the line that is count -1 lines forward from the current line, and any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Set to the current line + count-1.

Current column: The current column is set to the last display line column of the last non- <newline> in the line, or column position 1 if the line is empty.

The current column shall be adjusted to be on the last display line column of the last non- <newline> of the current line as subsequent commands change the current line, until a command changes the current column.  

Move to Matching Character

Synopsis:

 

%

 

If the character at the current position is not a parenthesis, bracket, or curly brace, search forward in the line to the first one of those characters. If no such character is found, it shall be an error.

The matching character shall be the parenthesis, bracket, or curly brace matching the parenthesis, bracket, or curly brace, respectively, that was at the current position or that was found on the current line.

Matching shall be determined as follows, for an open parenthesis:

1.
Set a counter to 1.

 

2.
Search forwards until a parenthesis is found or the end of the edit buffer is reached.

 

3.
If the end of the edit buffer is reached, it shall be an error.

 

4.
If an open parenthesis is found, increment the counter by 1.

 

5.
If a close parenthesis is found, decrement the counter by 1.

 

6.
If the counter is zero, the current character is the matching character.

 

Matching for a close parenthesis shall be equivalent, except that the search shall be backwards, from the starting character to the beginning of the buffer, a close parenthesis shall increment the counter by 1, and an open parenthesis shall decrement the counter by 1.

Matching for brackets and curly braces shall be equivalent, except that searching shall be done for open and close brackets or open and close curly braces. It is implementation-defined whether other characters are searched for and matched as well.

If used as a motion command:

1.
If the matching cursor was after the starting cursor in the edit buffer, and the starting cursor position was at or before the first non- <blank> non- <newline> in the starting line, and the matching cursor position was at or after the last non- <blank> non- <newline> in the matching line, the text region shall consist of the current line to the matching line, inclusive, and any text copied to a buffer shall be in line mode.

 

2.
If the matching cursor was before the starting cursor in the edit buffer, and the starting cursor position was at or after the last non- <blank> non- <newline> in the starting line, and the matching cursor position was at or before the first non- <blank> non- <newline> in the matching line, the text region shall consist of the current line to the matching line, inclusive, and any text copied to a buffer shall be in line mode.

 

3.
Otherwise, the text region shall consist of the starting character to the matching character, inclusive, and any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Set to the line where the matching character is located.

Current column: Set to the last column where any portion of the matching character is displayed.  

Repeat Substitution

Synopsis:

 

&

 

Repeat the previous substitution command. This command shall be equivalent to the ex & command with the current line as its addresses, and without options, count, or flags.  

Return to Previous Context at Beginning of Line

Synopsis:

 

' character

 

It shall be an error if there is no line in the edit buffer marked by character.

If used as a motion command:

1.
If the starting cursor is after the marked cursor, then the locations of the starting cursor and the marked cursor in the edit buffer shall be logically swapped.

 

2.
The text region shall consist of the starting line up to and including the marked line, and any text copied to a buffer shall be in line mode.

 

If not used as a motion command:

Current line: Set to the line referenced by the mark.

Current column: Set to non- <blank>.  

Return to Previous Context

Synopsis:

 

` character

 

It shall be an error if the marked line is no longer in the edit buffer. If the marked line no longer contains a character in the saved numbered character position, it shall be as if the marked position is the first non- <blank>.

If used as a motion command:

1.
It shall be an error if the marked cursor references the same character in the edit buffer as the starting cursor.

 

2.
If the starting cursor is after the marked cursor, then the locations of the starting cursor and the marked cursor in the edit buffer shall be logically swapped.

 

3.
If the starting line is empty or the starting cursor is at or before the first non- <blank> non- <newline> of the starting line, and the marked cursor line is empty or the marked cursor references the first character of the marked cursor line, the text region shall consist of all lines containing characters from the starting cursor to the line before the marked cursor line, inclusive, and any text copied to a buffer shall be in line mode.

 

4.
Otherwise, if the marked cursor line is empty or the marked cursor references a character at or before the first non- <blank> non- <newline> of the marked cursor line, the region of text shall be from the starting cursor to the last non- <newline> of the line before the marked cursor line, inclusive, and any text copied to a buffer shall be in character mode.

 

5.
Otherwise, the region of text shall be from the starting cursor (inclusive), to the marked cursor (exclusive), and any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Set to the line referenced by the mark.

Current column: Set to the last column in which any portion of the character referenced by the mark is displayed.  

Return to Previous Section

Synopsis:

 

[count] [[

 

Move the cursor backward through the edit buffer to the first character of the previous section boundary, count times.

If used as a motion command:

1.
If the starting cursor was at the first character of the starting line or the starting line was empty, and the first character of the boundary was the first character of the boundary line, the text region shall consist of the current line up to and including the line where the countth next boundary starts, and any text copied to a buffer shall be in line mode.

 

2.
If the boundary was the last line of the edit buffer or the last non- <newline> of the last line of the edit buffer, the text region shall consist of the last character in the edit buffer up to and including the starting character, and any text saved to a buffer shall be in character mode.

 

3.
Otherwise, the text region shall consist of the starting character up to but not including the first character in the countth next boundary, and any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Set to the line where the countth next boundary in the edit buffer starts.

Current column: Set to the last column in which any portion of the first character of the countth next boundary is displayed, or column position 1 if the line is empty.  

Move to Next Section

Synopsis:

 

[count] ]]

 

Move the cursor forward through the edit buffer to the first character of the next section boundary, count times.

If used as a motion command:

1.
If the starting cursor was at the first character of the starting line or the starting line was empty, and the first character of the boundary was the first character of the boundary line, the text region shall consist of the current line up to and including the line where the countth previous boundary starts, and any text copied to a buffer shall be in line mode.

 

2.
If the boundary was the first line of the edit buffer, the text region shall consist of the first character in the edit buffer up to but not including the starting character, and any text copied to a buffer shall be in character mode.

 

3.
Otherwise, the text region shall consist of the first character in the countth previous section boundary up to but not including the starting character, and any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Set to the line where the countth previous boundary in the edit buffer starts.

Current column: Set to the last column in which any portion of the first character of the countth previous boundary is displayed, or column position 1 if the line is empty.  

Move to First Non-<blank> Position on Current Line

Synopsis:

 

^

If used as a motion command:

1.
If the line has no non- <blank> non- <newline>s, or if the cursor is at the first non- <blank> non- <newline> of the line, it shall be an error.

 

2.
If the cursor is before the first non- <blank> non- <newline> of the line, the text region shall be comprised of the current character, up to, but not including, the first non- <blank> non- <newline> of the line.

 

3.
If the cursor is after the first non- <blank> non- <newline> of the line, the text region shall be from the character before the starting cursor up to and including the first non- <blank> non- <newline> of the line.

 

4.
Any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Unchanged.

Current column: Set to non- <blank>.  

Current and Line Above

Synopsis:

 

[count] _

 

If there are less than count -1 lines after the current line in the edit buffer, it shall be an error.

If used as a motion command:

1.
If count is less than 2, the text region shall be the current line.

 

2.
Otherwise, the text region shall include the starting line and the next count -1 lines.

 

3.
Any text copied to a buffer shall be in line mode.

 

If not used as a motion command:

Current line: Set to current line + count -1.

Current column: Set to non- <blank>.  

Move Back to Beginning of Sentence

Synopsis:

 

[count] (

 

Move backward to the beginning of a sentence. This command shall be equivalent to the [[ command, with the exception that sentence boundaries shall be used instead of section boundaries.  

Move Forward to Beginning of Sentence

Synopsis:

 

[count] )

 

Move forward to the beginning of a sentence. This command shall be equivalent to the ]] command, with the exception that sentence boundaries shall be used instead of section boundaries.  

Move Back to Preceding Paragraph

Synopsis:

 

[count] {

 

Move back to the beginning of the preceding paragraph. This command shall be equivalent to the [[ command, with the exception that paragraph boundaries shall be used instead of section boundaries.  

Move Forward to Next Paragraph

Synopsis:

 

[count] }

 

Move forward to the beginning of the next paragraph. This command shall be equivalent to the ]] command, with the exception that paragraph boundaries shall be used instead of section boundaries.  

Move to Specific Column Position

Synopsis:

 

[count] |

 

For the purposes of this command, lines that are too long for the current display and that have been folded shall be treated as having a single, 1-based, number of columns.

If there are less than count columns in which characters from the current line are displayed on the screen, count shall be adjusted to be the last column in which any portion of the line is displayed on the screen.

If used as a motion command:

1.
If the line is empty, or the cursor character is the same as the character on the countth column of the line, it shall be an error.

 

2.
If the cursor is before the countth column of the line, the text region shall be comprised of the current character, up to but not including the character on the countth column of the line.

 

3.
If the cursor is after the countth column of the line, the text region shall be from the character before the starting cursor up to and including the character on the countth column of the line.

 

4.
Any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Unchanged.

Current column: Set to the last column in which any portion of the character that is displayed in the count column of the line is displayed.  

Reverse Find Character

Synopsis:

 

[count] ,

 

If the last F, f, T, or t command was F, f, T, or t, this command shall be equivalent to an f, F, t, or T command, respectively, with the specified count and the same search character.

If there was no previous F, f, T, or t command, it shall be an error.  

Repeat

Synopsis:

 

[count] .

 

Repeat the last !, <, >, A, C, D, I, J, O, P, R, S, X, Y, a, c, d, i, o, p, r, s, x, y, or ~ command. It shall be an error if none of these commands have been executed. Commands (other than commands that enter text input mode) executed as a result of map expansions, shall not change the value of the last repeatable command.

Repeated commands with associated motion commands shall repeat the motion command as well; however, any specified count shall replace the count(s) that were originally specified to the repeated command or its associated motion command.

If the motion component of the repeated command is f, F, t, or T, the repeated command shall not set the remembered search character for the ; and , commands.

If the repeated command is p or P, and the buffer associated with that command was a numeric buffer named with a number less than 9, the buffer associated with the repeated command shall be set to be the buffer named by the name of the previous buffer logically incremented by 1.

If the repeated character is a text input command, the input text associated with that command is repeated literally:

*
Input characters are neither macro or abbreviation-expanded.

 

*
Input characters are not interpreted in any special way with the exception that <newline>, <carriage-return>, and <control>-T behave as described in Input Mode Commands in vi .

 

Current line: Set as described for the repeated command.

Current column: Set as described for the repeated command.  

Find Regular Expression

Synopsis:

 

/

 

If the input line contains no non- <newline>s, it shall be equivalent to a line containing only the last regular expression encountered. The enhanced regular expressions supported by vi are described in Regular Expressions in ex .

Otherwise, the line shall be interpreted as one or more regular expressions, optionally followed by an address offset or a vi z command.

If the regular expression is not the last regular expression on the line, or if a line offset or z command is specified, the regular expression shall be terminated by an unescaped '/' character, which shall not be used as part of the regular expression. If the regular expression is not the first regular expression on the line, it shall be preceded by zero or more <blank>s, a semicolon, zero or more <blank>s, and a leading '/' character, which shall not be interpreted as part of the regular expression. It shall be an error to precede any regular expression with any characters other than these.

Each search shall begin from the character after the first character of the last match (or, if it is the first search, after the cursor). If the wrapscan edit option is set, the search shall continue to the character before the starting cursor character; otherwise, to the end of the edit buffer. It shall be an error if any search fails to find a match, and an informational message to this effect shall be displayed.

An optional address offset (see Addressing in ex ) can be specified after the last regular expression by including a trailing '/' character after the regular expression and specifying the address offset. This offset will be from the line containing the match for the last regular expression specified. It shall be an error if the line offset would indicate a line address less than 1 or greater than the last line in the edit buffer. An address offset of zero shall be supported. It shall be an error to follow the address offset with any other characters than <blank>s.

If not used as a motion command, an optional z command (see Redraw Window ) can be specified after the last regular expression by including a trailing '/' character after the regular expression, zero or more <blank>s, a 'z' , zero or more <blank>s, an optional new window edit option value, zero or more <blank>s, and a location character. The effect shall be as if the z command was executed after the / command. It shall be an error to follow the z command with any other characters than <blank>s.

The remembered search direction shall be set to forward.

If used as a motion command:

1.
It shall be an error if the last match references the same character in the edit buffer as the starting cursor.

 

2.
If any address offset is specified, the last match shall be adjusted by the specified offset as described previously.

 

3.
If the starting cursor is after the last match, then the locations of the starting cursor and the last match in the edit buffer shall be logically swapped.

 

4.
If any address offset is specified, the text region shall consist of all lines containing characters from the starting cursor to the last match line, inclusive, and any text copied to a buffer shall be in line mode.

 

5.
Otherwise, if the starting line is empty or the starting cursor is at or before the first non- <blank> non- <newline> of the starting line, and the last match line is empty or the last match starts at the first character of the last match line, the text region shall consist of all lines containing characters from the starting cursor to the line before the last match line, inclusive, and any text copied to a buffer shall be in line mode.

 

6.
Otherwise, if the last match line is empty or the last match begins at a character at or before the first non- <blank> non- <newline> of the last match line, the region of text shall be from the current cursor to the last non- <newline> of the line before the last match line, inclusive, and any text copied to a buffer shall be in character mode.

 

7.
Otherwise, the region of text shall be from the current cursor (inclusive), to the first character of the last match (exclusive), and any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: If a match is found, set to the last matched line plus the address offset, if any; otherwise, unchanged.

Current column: Set to the last column on which any portion of the first character in the last matched string is displayed, if a match is found; otherwise, unchanged.  

Move to First Character in Line

Synopsis:

 

0  (zero)

 

Move to the first character on the current line. The character '0' shall not be interpreted as a command if it is immediately preceded by a digit.

If used as a motion command:

1.
If the cursor character is the first character in the line, it shall be an error.

 

2.
The text region shall be from the character before the cursor character up to and including the first character in the line.

 

3.
Any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Unchanged.

Current column: The last column in which any portion of the first character in the line is displayed, or if the line is empty, unchanged.  

Execute an ex Command

Synopsis:

 

:

 

Execute one or more ex commands.

If any portion of the screen other than the last line of the screen was overwritten by any ex command (except shell), vi shall display a message indicating that it is waiting for an input from the user, and shall then read a character. This action may also be taken for other, unspecified reasons.

If the next character entered is a ':' , another ex command shall be accepted and executed. Any other character shall cause the screen to be refreshed and vi shall return to command mode.

Current line: As specified for the ex command.

Current column: As specified for the ex command.  

Repeat Find

Synopsis:

 

[count] ;

 

This command shall be equivalent to the last F, f, T, or t command, with the specified count, and with the same search character used for the last F, f, T, or t command. If there was no previous F, f, T, or t command, it shall be an error.  

Shift Left

Synopsis:

 

[count] < motion

 

If the motion command is the < command repeated:

1.
If there are less than count -1 lines after the current line in the edit buffer, it shall be an error.

 

2.
The text region shall be from the current line, up to and including the next count -1 lines.

 

Shift any line in the text region specified by the count and motion command one shiftwidth (see the ex shiftwidth option) toward the start of the line, as described by the ex < command. The unshifted lines shall be copied to the unnamed buffer in line mode.

Current line: If the motion was from the current cursor position toward the end of the edit buffer, unchanged. Otherwise, set to the first line in the edit buffer that is part of the text region specified by the motion command.

Current column: Set to non- <blank>.  

Shift Right

Synopsis:

 

[count] > motion

 

If the motion command is the > command repeated:

1.
If there are less than count -1 lines after the current line in the edit buffer, it shall be an error.

 

2.
The text region shall be from the current line, up to and including the next count -1 lines.

 

Shift any line with characters in the text region specified by the count and motion command one shiftwidth (see the ex shiftwidth option) away from the start of the line, as described by the ex > command. The unshifted lines shall be copied into the unnamed buffer in line mode.

Current line: If the motion was from the current cursor position toward the end of the edit buffer, unchanged. Otherwise, set to the first line in the edit buffer that is part of the text region specified by the motion command.

Current column: Set to non- <blank>.  

Scan Backwards for Regular Expression

Synopsis:

 

?

 

Scan backwards; the ? command shall be equivalent to the / command (see Find Regular Expression ) with the following exceptions:

1.
The input prompt shall be a '?' .

 

2.
Each search shall begin from the character before the first character of the last match (or, if it is the first search, the character before the cursor character).

 

3.
The search direction shall be from the cursor toward the beginning of the edit buffer, and the wrapscan edit option shall affect whether the search wraps to the end of the edit buffer and continues.

 

4.
The remembered search direction shall be set to backward.

 

Execute

Synopsis:

 

@buffer

 

If the buffer is specified as @, the last buffer executed shall be used. If no previous buffer has been executed, it shall be an error.

Behave as if the contents of the named buffer were entered as standard input. After each line of a line-mode buffer, and all but the last line of a character mode buffer, behave as if a <newline> were entered as standard input.

If an error occurs during this process, an error message shall be written, and no more characters resulting from the execution of this command shall be processed.

If a count is specified, behave as if that count were entered as user input before the characters from the @ buffer were entered.

Current line: As specified for the individual commands.

Current column: As specified for the individual commands.  

Reverse Case

Synopsis:

 

[count] ~

 

Reverse the case of the current character and the next count -1 characters, such that lowercase characters that have uppercase counterparts shall be changed to uppercase characters, and uppercase characters that have lowercase counterparts shall be changed to lowercase characters, as prescribed by the current locale. No other characters shall be affected by this command.

If there are less than count -1 characters after the cursor in the edit buffer, count shall be adjusted to the number of characters after the cursor in the edit buffer minus 1.

For the purposes of this command, the next character after the last non- <newline> on the line shall be the next character in the edit buffer.

Current line: Set to the line including the ( count-1)th character after the cursor.

Current column: Set to the last column in which any portion of the ( count-1)th character after the cursor is displayed.  

Append

Synopsis:

 

[count] a

 

Enter text input mode after the current cursor position. No characters already in the edit buffer shall be affected by this command. A count shall cause the input text to be appended count -1 more times to the end of the input.

Current line/column: As specified for the text input commands (see Input Mode Commands in vi ).  

Append at End-of-Line

Synopsis:

 

[count] A

 

This command shall be equivalent to the vi command:

 

$ [ count ] a

(see Append ).  

Move Backward to Preceding Word

Synopsis:

 

[count] b

 

With the exception that words are used as the delimiter instead of bigwords, this command shall be equivalent to the B command.  

Move Backward to Preceding Bigword

Synopsis:

 

[count] B

 

If the edit buffer is empty or the cursor is on the first character of the edit buffer, it shall be an error. If less than count bigwords begin between the cursor and the start of the edit buffer, count shall be adjusted to the number of bigword beginnings between the cursor and the start of the edit buffer.

If used as a motion command:

1.
The text region shall be from the first character of the countth previous bigword beginning up to but not including the cursor character.

 

2.
Any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Set to the line containing the current column.

Current column: Set to the last column upon which any part of the first character of the countth previous bigword is displayed.  

Change

Synopsis:

 

[buffer][count] c motion

 

If the motion command is the c command repeated:

1.
The buffer text shall be in line mode.

 

2.
If there are less than count -1 lines after the current line in the edit buffer, it shall be an error.

 

3.
The text region shall be from the current line up to and including the next count -1 lines.

 

Otherwise, the buffer text mode and text region shall be as specified by the motion command.

The replaced text shall be copied into buffer, if specified, and into the unnamed buffer. If the text to be replaced contains characters from more than a single line, or the buffer text is in line mode, the replaced text shall be copied into the numeric buffers as well.

If the buffer text is in line mode:

1.
Any lines that contain characters in the region shall be deleted, and the editor shall enter text input mode at the beginning of a new line which shall replace the first line deleted.

 

2.
If the autoindent edit option is set, autoindent characters equal to the autoindent characters on the first line deleted shall be inserted as if entered by the user.

 

Otherwise, if characters from more than one line are in the region of text:

1.
The text shall be deleted.

 

2.
Any text remaining in the last line in the text region shall be appended to the first line in the region, and the last line in the region shall be deleted.

 

3.
The editor shall enter text input mode after the last character not deleted from the first line in the text region, if any; otherwise, on the first column of the first line in the region.

 

Otherwise:

1.
If the glyph for '$' is smaller than the region, the end of the region shall be marked with a '$' .

 

2.
The editor shall enter text input mode, overwriting the region of text.

 

Current line/column: As specified for the text input commands (see Input Mode Commands in vi ).  

Change to End-of-Line

Synopsis:

 

[buffer][count] C

 

This command shall be equivalent to the vi command:

 

[buffer][count] c$

See the c command.  

Delete

Synopsis:

 

[buffer][count] d motion

 

If the motion command is the d command repeated:

1.
The buffer text shall be in line mode.

 

2.
If there are less than count -1 lines after the current line in the edit buffer, it shall be an error.

 

3.
The text region shall be from the current line up to and including the next count -1 lines.

 

Otherwise, the buffer text mode and text region shall be as specified by the motion command.

If in open mode, and the current line is deleted, and the line remains on the display, an '@' character shall be displayed as the first glyph of that line.

Delete the region of text into buffer, if specified, and into the unnamed buffer. If the text to be deleted contains characters from more than a single line, or the buffer text is in line mode, the deleted text shall be copied into the numeric buffers, as well.

Current line: Set to the first text region line that appears in the edit buffer, unless that line has been deleted, in which case it shall be set to the last line in the edit buffer, or line 1 if the edit buffer is empty.

Current column:

1.
If the line is empty, set to column position 1.

 

2.
Otherwise, if the buffer text is in line mode or the motion was from the cursor toward the end of the edit buffer:
a.
If a character from the current line is displayed in the current column, set to the last column that displays any portion of that character.

 

b.
Otherwise, set to the last column in which any portion of any character in the line is displayed.

 

 

3.
Otherwise, if a character is displayed in the column that began the text region, set to the last column that displays any portion of that character.

 

4.
Otherwise, set to the last column in which any portion of any character in the line is displayed.

 

Delete to End-of-Line

Synopsis:

 

[buffer] D

 

Delete the text from the current position to the end of the current line; equivalent to the vi command:

 

[buffer] d$

 

Move to End-of-Word

Synopsis:

 

[count] e

 

With the exception that words are used instead of bigwords as the delimiter, this command shall be equivalent to the E command.  

Move to End-of-Bigword

Synopsis:

 

[count] E

 

If the edit buffer is empty it shall be an error. If less than count bigwords end between the cursor and the end of the edit buffer, count shall be adjusted to the number of bigword endings between the cursor and the end of the edit buffer.

If used as a motion command:

1.
The text region shall be from the last character of the countth next bigword up to and including the cursor character.

 

2.
Any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Set to the line containing the current column.

Current column: Set to the last column upon which any part of the last character of the countth next bigword is displayed.  

Find Character in Current Line (Forward)

Synopsis:

 

[count] f character

 

It shall be an error if count occurrences of the character do not occur after the cursor in the line.

If used as a motion command:

1.
The text range shall be from the cursor character up to and including the countth occurrence of the specified character after the cursor.

 

2.
Any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Unchanged.

Current column: Set to the last column in which any portion of the countth occurrence of the specified character after the cursor appears in the line.  

Find Character in Current Line (Reverse)

Synopsis:

 

[count] F character

 

It shall be an error if count occurrences of the character do not occur before the cursor in the line.

If used as a motion command:

1.
The text region shall be from the countth occurrence of the specified character before the cursor, up to, but not including the cursor character.

 

2.
Any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Unchanged.

Current column: Set to the last column in which any portion of the countth occurrence of the specified character before the cursor appears in the line.  

Move to Line

Synopsis:

 

[count] G

 

If count is not specified, it shall default to the last line of the edit buffer. If count is greater than the last line of the edit buffer, it shall be an error.

If used as a motion command:

1.
The text region shall be from the cursor line up to and including the specified line.

 

2.
Any text copied to a buffer shall be in line mode.

 

If not used as a motion command:

Current line: Set to count if count is specified; otherwise, the last line.

Current column: Set to non- <blank>.  

Move to Top of Screen

Synopsis:

 

[count] H

 

If the beginning of the line count greater than the first line of which any portion appears on the display does not exist, it shall be an error.

If used as a motion command:

1.
If in open mode, the text region shall be the current line.

 

2.
Otherwise, the text region shall be from the starting line up to and including (the first line of the display + count -1).

 

3.
Any text copied to a buffer shall be in line mode.

 

If not used as a motion command:

If in open mode, this command shall set the current column to non- <blank> and do nothing else.

Otherwise, it shall set the current line and current column as follows.

Current line: Set to (the first line of the display + count -1).

Current column: Set to non- <blank>.  

Insert Before Cursor

Synopsis:

 

[count] i

 

Enter text input mode before the current cursor position. No characters already in the edit buffer shall be affected by this command. A count shall cause the input text to be appended count -1 more times to the end of the input.

Current line/column: As specified for the text input commands (see Input Mode Commands in vi ).  

Insert at Beginning of Line

Synopsis:

 

[count] I

 

This command shall be equivalent to the vi command ^[ count] i.  

Join

Synopsis:

 

[count] J

 

If the current line is the last line in the edit buffer, it shall be an error.

This command shall be equivalent to the ex join command with no addresses, and an ex command count value of 1 if count was not specified or if a count of 1 was specified, and an ex command count value of count -1 for any other value of count, except that the current line and column shall be set as follows.

Current line: Unchanged.

Current column: The last column in which any portion of the character following the last character in the initial line is displayed, or the last non- <newline> in the line if no characters were appended.  

Move to Bottom of Screen

Synopsis:

 

[count] L

 

If the beginning of the line count less than the last line of which any portion appears on the display does not exist, it shall be an error.

If used as a motion command:

1.
If in open mode, the text region shall be the current line.

 

2.
Otherwise, the text region shall include all lines from the starting cursor line to (the last line of the display -( count -1)).

 

3.
Any text copied to a buffer shall be in line mode.

 

If not used as a motion command:

1.
If in open mode, this command shall set the current column to non- <blank> and do nothing else.

 

2.
Otherwise, it shall set the current line and current column as follows.

 

Current line: Set to (the last line of the display -( count -1)).

Current column: Set to non- <blank>.  

Mark Position

Synopsis:

 

m letter

 

This command shall be equivalent to the ex mark command with the specified character as an argument.  

Move to Middle of Screen

Synopsis:

 

M

 

The middle line of the display shall be calculated as follows:

 

(the top line of the display) + (((number of lines displayed) +1) /2) -1

If used as a motion command:

1.
If in open mode, the text region shall be the current line.

 

2.
Otherwise, the text region shall include all lines from the starting cursor line up to and including the middle line of the display.

 

3.
Any text copied to a buffer shall be in line mode.

 

If not used as a motion command:

If in open mode, this command shall set the current column to non- <blank> and do nothing else.

Otherwise, it shall set the current line and current column as follows.

Current line: Set to the middle line of the display.

Current column: Set to non- <blank>.  

Repeat Regular Expression Find (Forward)

Synopsis:

 

n

 

If the remembered search direction was forward, the n command shall be equivalent to the vi / command with no characters entered by the user. Otherwise, it shall be equivalent to the vi ? command with no characters entered by the user.

If the n command is used as a motion command for the ! command, the editor shall not enter text input mode on the last line on the screen, and shall behave as if the user entered a single '!' character as the text input.  

Repeat Regular Expression Find (Reverse)

Synopsis:

 

N

 

Scan for the next match of the last pattern given to / or ?, but in the reverse direction; this is the reverse of n.

If the remembered search direction was forward, the N command shall be equivalent to the vi ? command with no characters entered by the user. Otherwise, it shall be equivalent to the vi / command with no characters entered by the user. If the N command is used as a motion command for the ! command, the editor shall not enter text input mode on the last line on the screen, and shall behave as if the user entered a single ! character as the text input.  

Insert Empty Line Below

Synopsis:

 

o

 

Enter text input mode in a new line appended after the current line. A count shall cause the input text to be appended count -1 more times to the end of the already added text, each time starting on a new, appended line.

Current line/column: As specified for the text input commands (see Input Mode Commands in vi ).  

Insert Empty Line Above

Synopsis:

 

O

 

Enter text input mode in a new line inserted before the current line. A count shall cause the input text to be appended count -1 more times to the end of the already added text, each time starting on a new, appended line.

Current line/column: As specified for the text input commands (see Input Mode Commands in vi ).  

Put from Buffer Following

Synopsis:

 

[buffer] p

 

If no buffer is specified, the unnamed buffer shall be used.

If the buffer text is in line mode, the text shall be appended below the current line, and each line of the buffer shall become a new line in the edit buffer. A count shall cause the buffer text to be appended count -1 more times to the end of the already added text, each time starting on a new, appended line.

If the buffer text is in character mode, the text shall be appended into the current line after the cursor, and each line of the buffer other than the first and last shall become a new line in the edit buffer. A count shall cause the buffer text to be appended count -1 more times to the end of the already added text, each time starting after the last added character.

Current line: If the buffer text is in line mode, set the line to line +1; otherwise, unchanged.

Current column: If the buffer text is in line mode:

1.
If there is a non- <blank> in the first line of the buffer, set to the last column on which any portion of the first non- <blank> in the line is displayed.

 

2.
If there is no non- <blank> in the first line of the buffer, set to the last column on which any portion of the last non- <newline> in the first line of the buffer is displayed.

 

If the buffer text is in character mode:

1.
If the text in the buffer is from more than a single line, then set to the last column on which any portion of the first character from the buffer is displayed.

 

2.
Otherwise, if the buffer is the unnamed buffer, set to the last column on which any portion of the last character from the buffer is displayed.

 

3.
Otherwise, set to the first column on which any portion of the first character from the buffer is displayed.

 

Put from Buffer Before

Synopsis:

 

[buffer] P

 

If no buffer is specified, the unnamed buffer shall be used.

If the buffer text is in line mode, the text shall be inserted above the current line, and each line of the buffer shall become a new line in the edit buffer. A count shall cause the buffer text to be appended count -1 more times to the end of the already added text, each time starting on a new, appended line.

If the buffer text is in character mode, the text shall be inserted into the current line before the cursor, and each line of the buffer other than the first and last shall become a new line in the edit buffer. A count shall cause the buffer text to be appended count -1 more times to the end of the already added text, each time starting after the last added character.

Current line: Unchanged.

Current column: If the buffer text is in line mode:

1.
If there is a non- <blank> in the first line of the buffer, set to the last column on which any portion of that character is displayed.

 

2.
If there is no non- <blank> in the first line of the buffer, set to the last column on which any portion of the last non- <newline> in the first line of the buffer is displayed.

 

If the buffer text is in character mode:

1.
If the buffer is the unnamed buffer, set to the last column on which any portion of the last character from the buffer is displayed.

 

2.
Otherwise, set to the first column on which any portion of the first character from the buffer is displayed.

 

Enter ex Mode

Synopsis:

 

Q

 

Leave visual or open mode and enter ex command mode.

Current line: Unchanged.

Current column: Unchanged.  

Replace Character

Synopsis:

 

[count] r character

 

Replace the count characters at and after the cursor with the specified character. If there are less than count non- <newline>s at and after the cursor on the line, it shall be an error.

If character is <control>-V, any next character other than the <newline> shall be stripped of any special meaning and used as a literal character.

If character is <ESC>, no replacement shall be made and the current line and current column shall be unchanged.

If character is <carriage-return> or <newline>, count new lines shall be appended to the current line. All but the last of these lines shall be empty. count characters at and after the cursor shall be discarded, and any remaining characters after the cursor in the current line shall be moved to the last of the new lines. If the autoindent edit option is set, they shall be preceded by the same number of autoindent characters found on the line from which the command was executed.

Current line: Unchanged unless the replacement character is a <carriage-return> or <newline>, in which case it shall be set to line + count.

Current column: Set to the last column position on which a portion of the last replaced character is displayed, or if the replacement character caused new lines to be created, set to non- <blank>.  

Replace Characters

Synopsis:

 

R

 

Enter text input mode at the current cursor position possibly replacing text on the current line. A count shall cause the input text to be appended count -1 more times to the end of the input.

Current line/column: As specified for the text input commands (see Input Mode Commands in vi ).  

Substitute Character

Synopsis:

 

[buffer][count] s

 

This command shall be equivalent to the vi command:

 

[buffer][count] c<space>

 

Substitute Lines

Synopsis:

 

[buffer][count] S

 

This command shall be equivalent to the vi command:

 

[buffer][count] c_

 

Move Cursor to Before Character (Forward)

Synopsis:

 

[count] t character

 

It shall be an error if count occurrences of the character do not occur after the cursor in the line.

If used as a motion command:

1.
The text region shall be from the cursor up to but not including the countth occurrence of the specified character after the cursor.

 

2.
Any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Unchanged.

Current column: Set to the last column in which any portion of the character before the countth occurrence of the specified character after the cursor appears in the line.  

Move Cursor to After Character (Reverse)

Synopsis:

 

[count] T character

 

It shall be an error if count occurrences of the character do not occur before the cursor in the line.

If used as a motion command:

1.
If the character before the cursor is the specified character, it shall be an error.

 

2.
The text region shall be from the character before the cursor up to but not including the countth occurrence of the specified character before the cursor.

 

3.
Any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

Current line: Unchanged.

Current column: Set to the last column in which any portion of the character after the countth occurrence of the specified character before the cursor appears in the line.  

Undo

Synopsis:

 

u

 

This command shall be equivalent to the ex undo command except that the current line and current column shall be set as follows:

Current line: Set to the first line added or changed if any; otherwise, move to the line preceding any deleted text if one exists; otherwise, move to line 1.

Current column: If undoing an ex command, set to the first non- <blank>.

Otherwise, if undoing a text input command:

1.
If the command was a C, c, O, o, R, S, or s command, the current column shall be set to the value it held when the text input command was entered.

 

2.
Otherwise, set to the last column in which any portion of the first character after the deleted text is displayed, or, if no non- <newline>s follow the text deleted from this line, set to the last column in which any portion of the last non- <newline> in the line is displayed, or 1 if the line is empty.

 

Otherwise, if a single line was modified (that is, not added or deleted) by the u command:

1.
If text was added or changed, set to the last column in which any portion of the first character added or changed is displayed.

 

2.
If text was deleted, set to the last column in which any portion of the first character after the deleted text is displayed, or, if no non- <newline>s follow the deleted text, set to the last column in which any portion of the last non- <newline> in the line is displayed, or 1 if the line is empty.

 

Otherwise, set to non- <blank>.  

Undo Current Line

Synopsis:

 

U

 

Restore the current line to its state immediately before the most recent time that it became the current line.

Current line: Unchanged.

Current column: Set to the first column in the line in which any portion of the first character in the line is displayed.  

Move to Beginning of Word

Synopsis:

 

[count] w

 

With the exception that words are used as the delimiter instead of bigwords, this command shall be equivalent to the W command.  

Move to Beginning of Bigword

Synopsis:

 

[count] W

 

If the edit buffer is empty, it shall be an error. If there are less than count bigwords between the cursor and the end of the edit buffer, count shall be adjusted to move the cursor to the last bigword in the edit buffer.

If used as a motion command:

1.
If the associated command is c, count is 1, and the cursor is on a <blank>, the region of text shall be the current character and no further action shall be taken.

 

2.
If there are less than count bigwords between the cursor and the end of the edit buffer, then the command shall succeed, and the region of text shall include the last character of the edit buffer.

 

3.
If there are <blank>s or an end-of-line that precede the countth bigword, and the associated command is c, the region of text shall be up to and including the last character before the preceding <blank>s or end-of-line.

 

4.
If there are <blank>s or an end-of-line that precede the bigword, and the associated command is d or y, the region of text shall be up to and including the last <blank> before the start of the bigword or end-of-line.

 

5.
Any text copied to a buffer shall be in character mode.

 

If not used as a motion command:

1.
If the cursor is on the last character of the edit buffer, it shall be an error.

 

Current line: Set to the line containing the current column.

Current column: Set to the last column in which any part of the first character of the countth next bigword is displayed.  

Delete Character at Cursor

Synopsis:

 

[buffer][count] x

 

Delete the count characters at and after the current character into buffer, if specified, and into the unnamed buffer.

If the line is empty, it shall be an error. If there are less than count non- <newline>s at and after the cursor on the current line, count shall be adjusted to the number of non- <newline>s at and after the cursor.

Current line: Unchanged.

Current column: If the line is empty, set to column position 1. Otherwise, if there were count or less non- <newline>s at and after the cursor on the current line, set to the last column that displays any part of the last non- <newline> of the line. Otherwise, unchanged.  

Delete Character Before Cursor

Synopsis:

 

[buffer][count] X

 

Delete the count characters before the current character into buffer, if specified, and into the unnamed buffer.

If there are no characters before the current character on the current line, it shall be an error. If there are less than count previous characters on the current line, count shall be adjusted to the number of previous characters on the line.

Current line: Unchanged.

Current column: Set to (current column - the width of the deleted characters).  

Yank

Synopsis:

 

[buffer][count] y motion

 

Copy (yank) the region of text into buffer, if specified, and into the unnamed buffer.

If the motion command is the y command repeated:

1.
The buffer shall be in line mode.

 

2.
If there are less than count -1 lines after the current line in the edit buffer, it shall be an error.

 

3.
The text region shall be from the current line up to and including the next count -1 lines.

 

Otherwise, the buffer text mode and text region shall be as specified by the motion command.

Current line: If the motion was from the current cursor position toward the end of the edit buffer, unchanged. Otherwise, set to the first line in the edit buffer that is part of the text region specified by the motion command.

Current column:

1.
If the motion was from the current cursor position toward the end of the edit buffer, unchanged.

 

2.
Otherwise, if the current line is empty, set to column position 1.

 

3.
Otherwise, set to the last column that displays any part of the first character in the file that is part of the text region specified by the motion command.

 

Yank Current Line

Synopsis:

 

[buffer][count] Y

 

This command shall be equivalent to the vi command:

 

[buffer][count] y_

 

Redraw Window

If in open mode, the z command shall have the Synopsis:

Synopsis:

 

[count] z

 

If count is not specified, it shall default to the window edit option -1. The z command shall be equivalent to the ex z command, with a type character of = and a count of count -2, except that the current line and current column shall be set as follows, and the window edit option shall not be affected. If the calculation for the count argument would result in a negative number, the count argument to the ex z command shall be zero. A blank line shall be written after the last line is written.

Current line: Unchanged.

Current column: Unchanged.

If not in open mode, the z command shall have the following Synopsis:

Synopsis:

 

[line] z [count] character

 

If line is not specified, it shall default to the current line. If line is specified, but is greater than the number of lines in the edit buffer, it shall default to the number of lines in the edit buffer.

If count is specified, the value of the window edit option shall be set to count (as described in the ex window command), and the screen shall be redrawn.

line shall be placed as specified by the following characters:

<newline>, <carriage-return>

Place the beginning of the line on the first line of the display.

.
Place the beginning of the line in the center of the display. The middle line of the display shall be calculated as described for the M command.
-
Place an unspecified portion of the line on the last line of the display.
+
If line was specified, equivalent to the <newline> case. If line was not specified, display a screen where the first line of the display shall be (current last line) +1. If there are no lines after the last line in the display, it shall be an error.
^
If line was specified, display a screen where the last line of the display shall contain an unspecified portion of the first line of a display that had an unspecified portion of the specified line on the last line of the display. If this calculation results in a line before the beginning of the edit buffer, display the first screen of the edit buffer.

Otherwise, display a screen where the last line of the display shall contain an unspecified portion of (current first line -1). If this calculation results in a line before the beginning of the edit buffer, it shall be an error.

 

Current line: If line and the '^' character were specified:

1.
If the first screen was displayed as a result of the command attempting to display lines before the beginning of the edit buffer: if the first screen was already displayed, unchanged; otherwise, set to (current first line -1).

 

2.
Otherwise, set to the last line of the display.

 

If line and the '+' character were specified, set to the first line of the display.

Otherwise, if line was specified, set to line.

Otherwise, unchanged.

Current column: Set to non- <blank>.  

Exit

Synopsis:

 

ZZ

 

This command shall be equivalent to the ex xit command with no addresses, trailing !, or filename (see the ex xit command).  

Input Mode Commands in vi

In text input mode, the current line shall consist of zero or more of the following categories, plus the terminating <newline>:

1.
Characters preceding the text input entry point

Characters in this category shall not be modified during text input mode.

 

2.
autoindent characters

autoindent characters shall be automatically inserted into each line that is created in text input mode, either as a result of entering a <newline> or <carriage-return> while in text input mode, or as an effect of the command itself; for example, O or o (see the ex autoindent command), as if entered by the user.

It shall be possible to erase autoindent characters with the <control>-D command; it is unspecified whether they can be erased by <control>-H, <control>-U, and <control>-W characters. Erasing any autoindent character turns the glyph into erase-columns and deletes the character from the edit buffer, but does not change its representation on the screen.

 

3.
Text input characters

Text input characters are the characters entered by the user. Erasing any text input character turns the glyph into erase-columns and deletes the character from the edit buffer, but does not change its representation on the screen.

Each text input character entered by the user (that does not have a special meaning) shall be treated as follows:

a.
The text input character shall be appended to the last character in the edit buffer from the first, second, or third categories.

 

b.
If there are no erase-columns on the screen, the text input command was the R command, and characters in the fifth category from the original line follow the cursor, the next such character shall be deleted from the edit buffer. If the slowopen edit option is not set, the corresponding glyph on the screen shall become erase-columns.

 

c.
If there are erase-columns on the screen, as many columns as they occupy, or as are necessary, shall be overwritten to display the text input character. (If only part of a multi-column glyph is overwritten, the remainder shall be left on the screen, and continue to be treated as erase-columns; it is unspecified whether the remainder of the glyph is modified in any way.)

 

d.
If additional display line columns are needed to display the text input character:
1.
If the slowopen edit option is set, the text input characters shall be displayed on subsequent display line columns, overwriting any characters displayed in those columns.

 

2.
Otherwise, any characters currently displayed on or after the column on the display line where the text input character is to be displayed shall be pushed ahead the number of display line columns necessary to display the rest of the text input character.

 

 

 

4.
Erase-columns

Erase-columns are not logically part of the edit buffer, appearing only on the screen, and may be overwritten on the screen by subsequent text input characters. When text input mode ends, all erase-columns shall no longer appear on the screen.

Erase-columns are initially the region of text specified by the c command (see Change ); however, erasing autoindent or text input characters causes the glyphs of the erased characters to be treated as erase-columns.

 

5.
Characters following the text region for the c command, or the text input entry point for all other commands

Characters in this category shall not be modified during text input mode, except as specified in category 3.b. for the R text input command, or as <blank>s deleted when a <newline> or <carriage-return> is entered.

 

It is unspecified whether it is an error to attempt to erase past the beginning of a line that was created by the entry of a <newline> or <carriage-return> during text input mode. If it is not an error, the editor shall behave as if the erasing character was entered immediately after the last text input character entered on the previous line, and all of the non- <newline>s on the current line shall be treated as erase-columns.

When text input mode is entered, or after a text input mode character is entered (except as specified for the special characters below), the cursor shall be positioned as follows:

1.
On the first column that displays any part of the first erase-column, if one exists

 

2.
Otherwise, if the slowopen edit option is set, on the first display line column after the last character in the first, second, or third categories, if one exists

 

3.
Otherwise, the first column that displays any part of the first character in the fifth category, if one exists

 

4.
Otherwise, the display line column after the last character in the first, second, or third categories, if one exists

 

5.
Otherwise, on column position 1

 

The characters that are updated on the screen during text input mode are unspecified, other than that the last text input character shall always be updated, and, if the slowopen edit option is not set, the current cursor character shall always be updated.

The following specifications are for command characters entered during text input mode.  

NUL

Synopsis:

 

NUL

 

If the first character of the text input is a NUL, the most recently input text shall be input as if entered by the user, and then text input mode shall be exited. The text shall be input literally; that is, characters are neither macro or abbreviation expanded, nor are any characters interpreted in any special manner. It is unspecified whether implementations shall support more than 256 bytes of remembered input text.  

<control>-D

Synopsis:

 

<control>-D

 

The <control>-D character shall have no special meaning when in text input mode for a line-oriented command (see Command Descriptions in vi ).

This command need not be supported on block-mode terminals.

If the cursor does not follow an autoindent character, or an autoindent character and a '0' or '^' character:

1.
If the cursor is in column position 1, the <control>-D character shall be discarded and no further action taken.

 

2.
Otherwise, the <control>-D character shall have no special meaning.

 

If the last input character was a '0' , the cursor shall be moved to column position 1.

Otherwise, if the last input character was a '^' , the cursor shall be moved to column position 1. In addition, the autoindent level for the next input line shall be derived from the same line from which the autoindent level for the current input line was derived.

Otherwise, the cursor shall be moved back to the column after the previous shiftwidth (see the ex shiftwidth command) boundary.

All of the glyphs on columns between the starting cursor position and (inclusively) the ending cursor position shall become erase-columns as described in Input Mode Commands in vi .

Current line: Unchanged.

Current column: Set to 1 if the <control>-D was preceded by a '^' or '0' ; otherwise, set to (column -1) -((column -2) % shiftwidth).  

<control>-H

Synopsis:

 

<control>-H

 

If in text input mode for a line-oriented command, and there are no characters to erase, text input mode shall be terminated, no further action shall be done for this command, and the current line and column shall be unchanged.

If there are characters other than autoindent characters that have been input on the current line before the cursor, the cursor shall move back one character.

Otherwise, if there are autoindent characters on the current line before the cursor, it is implementation-defined whether the <control>-H command is an error or if the cursor moves back one autoindent character.

Otherwise, if the cursor is in column position 1 and there are previous lines that have been input, it is implementation-defined whether the <control>-H command is an error or if it is equivalent to entering <control>-H after the last input character on the previous input line.

Otherwise, it shall be an error.

All of the glyphs on columns between the starting cursor position and (inclusively) the ending cursor position shall become erase-columns as described in Input Mode Commands in vi .

The current erase character (see stty) shall cause an equivalent action to the <control>-H command, unless the previously inserted character was a backslash, in which case it shall be as if the literal current erase character had been inserted instead of the backslash.

Current line: Unchanged, unless previously input lines are erased, in which case it shall be set to line -1.

Current column: Set to the first column that displays any portion of the character backed up over.  

<newline>

Synopsis:

 

<newline>

<carriage-return>

<control>-J

<control>-M

 

If input was part of a line-oriented command, text input mode shall be terminated and the command shall continue execution with the input provided.

Otherwise, terminate the current line. If there are no characters other than autoindent characters on the line, all characters on the line shall be discarded. Otherwise, it is unspecified whether the autoindent characters in the line are modified by entering these characters.

Continue text input mode on a new line appended after the current line. If the slowopen edit option is set, the lines on the screen below the current line shall not be pushed down, but the first of them shall be cleared and shall appear to be overwritten. Otherwise, the lines of the screen below the current line shall be pushed down.

If the autoindent edit option is set, an appropriate number of autoindent characters shall be added as a prefix to the line as described by the ex autoindent edit option.

All columns after the cursor that are erase-columns (as described in Input Mode Commands in vi ) shall be discarded.

If the autoindent edit option is set, all <blank>s immediately following the cursor shall be discarded.

All remaining characters after the cursor shall be transferred to the new line, positioned after any autoindent characters.

Current line: Set to current line +1.

Current column: Set to the first column that displays any portion of the first character after the autoindent characters on the new line, if any, or the first column position after the last autoindent character, if any, or column position 1.  

<control>-T

Synopsis:

 

<control>-T

 

The <control>-T character shall have no special meaning when in text input mode for a line-oriented command (see Command Descriptions in vi ).

This command need not be supported on block-mode terminals.

Behave as if the user entered the minimum number of <blank>s necessary to move the cursor forward to the column position after the next shiftwidth (see the ex shiftwidth command) boundary.

Current line: Unchanged.

Current column: Set to column + shiftwidth - ((column -1) % shiftwidth).  

<control>-U

Synopsis:

 

<control>-U

 

If there are characters other than autoindent characters that have been input on the current line before the cursor, the cursor shall move to the first character input after the autoindent characters.

Otherwise, if there are autoindent characters on the current line before the cursor, it is implementation-defined whether the <control>-U command is an error or if the cursor moves to the first column position on the line.

Otherwise, if the cursor is in column position 1 and there are previous lines that have been input, it is implementation-defined whether the <control>-U command is an error or if it is equivalent to entering <control>-U after the last input character on the previous input line.

Otherwise, it shall be an error.

All of the glyphs on columns between the starting cursor position and (inclusively) the ending cursor position shall become erase-columns as described in Input Mode Commands in vi .

The current kill character (see stty) shall cause an equivalent action to the <control>-U command, unless the previously inserted character was a backslash, in which case it shall be as if the literal current kill character had been inserted instead of the backslash.

Current line: Unchanged, unless previously input lines are erased, in which case it shall be set to line -1.

Current column: Set to the first column that displays any portion of the last character backed up over.  

<control>-V

Synopsis:

 

<control>-V

<control>-Q

 

Allow the entry of any subsequent character, other than <control>-J or the <newline>, as a literal character, removing any special meaning that it may have to the editor in text input mode. If a <control>-V or <control>-Q is entered before a <control>-J or <newline>, the <control>-V or <control>-Q character shall be discarded, and the <control>-J or <newline> shall behave as described in the <newline> command character during input mode.

For purposes of the display only, the editor shall behave as if a '^' character was entered, and the cursor shall be positioned as if overwriting the '^' character. When a subsequent character is entered, the editor shall behave as if that character was entered instead of the original <control>-V or <control>-Q character.

Current line: Unchanged.

Current column: Unchanged.  

<control>-W

Synopsis:

 

<control>-W

 

If there are characters other than autoindent characters that have been input on the current line before the cursor, the cursor shall move back over the last word preceding the cursor (including any <blank>s between the end of the last word and the current cursor); the cursor shall not move to before the first character after the end of any autoindent characters.

Otherwise, if there are autoindent characters on the current line before the cursor, it is implementation-defined whether the <control>-W command is an error or if the cursor moves to the first column position on the line.

Otherwise, if the cursor is in column position 1 and there are previous lines that have been input, it is implementation-defined whether the <control>-W command is an error or if it is equivalent to entering <control>-W after the last input character on the previous input line.

Otherwise, it shall be an error.

All of the glyphs on columns between the starting cursor position and (inclusively) the ending cursor position shall become erase-columns as described in Input Mode Commands in vi .

Current line: Unchanged, unless previously input lines are erased, in which case it shall be set to line -1.

Current column: Set to the first column that displays any portion of the last character backed up over.  

<ESC>

Synopsis:

 

<ESC>

 

If input was part of a line-oriented command:

1.
If interrupt was entered, text input mode shall be terminated and the editor shall return to command mode. The terminal shall be alerted.

 

2.
If <ESC> was entered, text input mode shall be terminated and the command shall continue execution with the input provided.

 

Otherwise, terminate text input mode and return to command mode.

Any autoindent characters entered on newly created lines that have no other non- <newline>s shall be deleted.

Any leading autoindent and <blank>s on newly created lines shall be rewritten to be the minimum number of <blank>s possible.

The screen shall be redisplayed as necessary to match the contents of the edit buffer.

Current line: Unchanged.

Current column:

1.
If there are text input characters on the current line, the column shall be set to the last column where any portion of the last text input character is displayed.

 

2.
Otherwise, if a character is displayed in the current column, unchanged.

 

3.
Otherwise, set to column position 1.

 

EXIT STATUS

The following exit values shall be returned:

 0
Successful completion.
>0
An error occurred.

 

 

CONSEQUENCES OF ERRORS

When any error is encountered and the standard input is not a terminal device file, vi shall not write the file or return to command or text input mode, and shall terminate with a non-zero exit status.

Otherwise, when an unrecoverable error is encountered it shall be equivalent to a SIGHUP asynchronous event.

Otherwise, when an error is encountered, the editor shall behave as specified in Command Descriptions in vi .

The following sections are informative.  

APPLICATION USAGE

None.  

EXAMPLES

None.  

RATIONALE

See the RATIONALE for ex for more information on vi. Major portions of the vi utility specification point to ex to avoid inadvertent divergence. While ex and vi have historically been implemented as a single utility, this is not required by IEEE Std 1003.1-2001.

It is recognized that portions of vi would be difficult, if not impossible, to implement satisfactorily on a block-mode terminal, or a terminal without any form of cursor addressing, thus it is not a mandatory requirement that such features should work on all terminals. It is the intention, however, that a vi implementation should provide the full set of capabilities on all terminals capable of supporting them.

Historically, vi exited immediately if the standard input was not a terminal. IEEE Std 1003.1-2001 permits, but does not require, this behavior. An end-of-file condition is not equivalent to an end-of-file character. A common end-of-file character, <control>-D, is historically a vi command.

The text in the STDOUT section reflects the usage of the verb display in this section; some implementations of vi use standard output to write to the terminal, but IEEE Std 1003.1-2001 does not require that to be the case.

Historically, implementations reverted to open mode if the terminal was incapable of supporting full visual mode. IEEE Std 1003.1-2001 requires this behavior. Historically, the open mode of vi behaved roughly equivalently to the visual mode, with the exception that only a single line from the edit buffer (one "buffer line") was kept current at any time. This line was normally displayed on the next-to-last line of a terminal with cursor addressing (and the last line performed its normal visual functions for line-oriented commands and messages). In addition, some few commands behaved differently in open mode than in visual mode. IEEE Std 1003.1-2001 requires conformance to historical practice.

Historically, ex and vi implementations have expected text to proceed in the usual European/Latin order of left to right, top to bottom. There is no requirement in IEEE Std 1003.1-2001 that this be the case. The specification was deliberately written using words like "before", "after", "first", and "last" in order to permit implementations to support the natural text order of the language.

Historically, lines past the end of the edit buffer were marked with single tilde ( '~' ) characters; that is, if the one-based display was 20 lines in length, and the last line of the file was on line one, then lines 2-20 would contain only a single '~' character.

Historically, the vi editor attempted to display only complete lines at the bottom of the screen (it did display partial lines at the top of the screen). If a line was too long to fit in its entirety at the bottom of the screen, the screen lines where the line would have been displayed were displayed as single '@' characters, instead of displaying part of the line. IEEE Std 1003.1-2001 permits, but does not require, this behavior. Implementations are encouraged to attempt always to display a complete line at the bottom of the screen when doing scrolling or screen positioning by buffer lines.

Historically, lines marked with '@' were also used to minimize output to dumb terminals over slow lines; that is, changes local to the cursor were updated, but changes to lines on the screen that were not close to the cursor were simply marked with an '@' sign instead of being updated to match the current text. IEEE Std 1003.1-2001 permits, but does not require this feature because it is used ever less frequently as terminals become smarter and connections are faster.  

Initialization in ex and vi

Historically, vi always had a line in the edit buffer, even if the edit buffer was "empty". For example:

1.
The ex command = executed from visual mode wrote "1" when the buffer was empty.

 

2.
Writes from visual mode of an empty edit buffer wrote files of a single character (a <newline>), while writes from ex mode of an empty edit buffer wrote empty files.

 

3.
Put and read commands into an empty edit buffer left an empty line at the top of the edit buffer.

 

For consistency, IEEE Std 1003.1-2001 does not permit any of these behaviors.

Historically, vi did not always return the terminal to its original modes; for example, ICRNL was modified if it was not originally set. IEEE Std 1003.1-2001 does not permit this behavior.  

Command Descriptions in vi

Motion commands are among the most complicated aspects of vi to describe. With some exceptions, the text region and buffer type effect of a motion command on a vi command are described on a case-by-case basis. The descriptions of text regions in IEEE Std 1003.1-2001 are not intended to imply direction; that is, an inclusive region from line n to line n+5 is identical to a region from line n+5 to line n. This is of more than academic interest-movements to marks can be in either direction, and, if the wrapscan option is set, so can movements to search points. Historically, lines are always stored into buffers in text order; that is, from the start of the edit buffer to the end. IEEE Std 1003.1-2001 requires conformance to historical practice.

Historically, command counts were applied to any associated motion, and were multiplicative to any supplied motion count. For example, 2cw is the same as c2w, and 2c3w is the same as c6w. IEEE Std 1003.1-2001 requires this behavior. Historically, vi commands that used bigwords, words, paragraphs, and sentences as objects treated groups of empty lines, or lines that contained only <blank>s, inconsistently. Some commands treated them as a single entity, while others treated each line separately. For example, the w, W, and B commands treated groups of empty lines as individual words; that is, the command would move the cursor to each new empty line. The e and E commands treated groups of empty lines as a single word; that is, the first use would move past the group of lines. The b command would just beep at the user, or if done from the start of the line as a motion command, fail in unexpected ways. If the lines contained only (or ended with) <blank>s, the w and W commands would just beep at the user, the E and e commands would treat the group as a single word, and the B and b commands would treat the lines as individual words. For consistency and simplicity of specification, IEEE Std 1003.1-2001 requires that all vi commands treat groups of empty or blank lines as a single entity, and that movement through lines ending with <blank>s be consistent with other movements.

Historically, vi documentation indicated that any number of double quotes were skipped after punctuation marks at sentence boundaries; however, implementations only skipped single quotes. IEEE Std 1003.1-2001 requires both to be skipped.

Historically, the first and last characters in the edit buffer were word boundaries. This historical practice is required by IEEE Std 1003.1-2001.

Historically, vi attempted to update the minimum number of columns on the screen possible, which could lead to misleading information being displayed. IEEE Std 1003.1-2001 makes no requirements other than that the current character being entered is displayed correctly, leaving all other decisions in this area up to the implementation.

Historically, lines were arbitrarily folded between columns of any characters that required multiple column positions on the screen, with the exception of tabs, which terminated at the right-hand margin. IEEE Std 1003.1-2001 permits the former and requires the latter. Implementations that do not arbitrarily break lines between columns of characters that occupy multiple column positions should not permit the cursor to rest on a column that does not contain any part of a character.

The historical vi had a problem in that all movements were by buffer lines, not by display or screen lines. This is often the right thing to do; for example, single line movements, such as j or k, should work on buffer lines. Commands like dj, or j., where . is a change command, only make sense for buffer lines. It is not, however, the right thing to do for screen motion or scrolling commands like <control>-D, <control>-F, and H. If the window is fairly small, using buffer lines in these cases can result in completely random motion; for example, 1 <control>-D can result in a completely changed screen, without any overlap. This is clearly not what the user wanted. The problem is even worse in the case of the H, L, and M commands-as they position the cursor at the first non- <blank> of the line, they may all refer to the same location in large lines, and will result in no movement at all.

In addition, if the line is larger than the screen, using buffer lines can make it impossible to display parts of the line-there are not any commands that do not display the beginning of the line in historical vi, and if both the beginning and end of the line cannot be on the screen at the same time, the user suffers. Finally, the page and half-page scrolling commands historically moved to the first non- <blank> in the new line. If the line is approximately the same size as the screen, this is inadequate because the cursor before and after a <control>-D command will refer to the same location on the screen.

Implementations of ex and vi exist that do not have these problems because the relevant commands ( <control>-B, <control>-D, <control>-F, <control>-U, <control>-Y, <control>-E, H, L, and M) operate on display (screen) lines, not (edit) buffer lines.

IEEE Std 1003.1-2001 does not permit this behavior by default because the standard developers believed that users would find it too confusing. However, historical practice has been relaxed. For example, ex and vi historically attempted, albeit sometimes unsuccessfully, to never put part of a line on the last lines of a screen; for example, if a line would not fit in its entirety, no part of the line was displayed, and the screen lines corresponding to the line contained single '@' characters. This behavior is permitted, but not required by IEEE Std 1003.1-2001, so that it is possible for implementations to support long lines in small screens more reasonably without changing the commands to be oriented to the display (instead of oriented to the buffer). IEEE Std 1003.1-2001 also permits implementations to refuse to edit any edit buffer containing a line that will not fit on the screen in its entirety.

The display area (for example, the value of the window edit option) has historically been "grown", or expanded, to display new text when local movements are done in displays where the number of lines displayed is less than the maximum possible. Expansion has historically been the first choice, when the target line is less than the maximum possible expansion value away. Scrolling has historically been the next choice, done when the target line is less than half a display away, and otherwise, the screen was redrawn. There were exceptions, however, in that ex commands generally always caused the screen to be redrawn. IEEE Std 1003.1-2001 does not specify a standard behavior because there may be external issues, such as connection speed, the number of characters necessary to redraw as opposed to scroll, or terminal capabilities that implementations will have to accommodate.

The current line in IEEE Std 1003.1-2001 maps one-to-one to a buffer line in the file. The current column does not. There are two different column values that are described by IEEE Std 1003.1-2001. The first is the current column value as set by many of the vi commands. This value is remembered for the lifetime of the editor. The second column value is the actual position on the screen where the cursor rests. The two are not always the same. For example, when the cursor is backed by a multi-column character, the actual cursor position on the screen has historically been the last column of the character in command mode, and the first column of the character in input mode.

Commands that set the current line, but that do not set the current cursor value (for example, j and k) attempt to get as close as possible to the remembered column position, so that the cursor tends to restrict itself to a vertical column as the user moves around in the edit buffer. IEEE Std 1003.1-2001 requires conformance to historical practice, requiring that the display location of the cursor on the display line be adjusted from the current column value as necessary to support this historical behavior.

Historically, only a single line (and for some terminals, a single line minus 1 column) of characters could be entered by the user for the line-oriented commands; that is, :, !, /, or ?. IEEE Std 1003.1-2001 permits, but does not require, this limitation.

Historically, "soft" errors in vi caused the terminal to be alerted, but no error message was displayed. As a general rule, no error message was displayed for errors in command execution in vi, when the error resulted from the user attempting an invalid or impossible action, or when a searched-for object was not found. Examples of soft errors included h at the left margin, <control>-B or [[ at the beginning of the file, 2G at the end of the file, and so on. In addition, errors such as %, ]], }, ), N, n, f, F, t, and T failing to find the searched-for object were soft as well. Less consistently, / and ? displayed an error message if the pattern was not found, /, ?, N, and n displayed an error message if no previous regular expression had been specified, and ; did not display an error message if no previous f, F, t, or T command had occurred. Also, behavior in this area might reasonably be based on a runtime evaluation of the speed of a network connection. Finally, some implementations have provided error messages for soft errors in order to assist naive users, based on the value of a verbose edit option. IEEE Std 1003.1-2001 does not list specific errors for which an error message shall be displayed. Implementations should conform to historical practice in the absence of any strong reason to diverge.  

Page Backwards

The <control>-B and <control>-F commands historically considered it an error to attempt to page past the beginning or end of the file, whereas the <control>-D and <control>-U commands simply moved to the beginning or end of the file. For consistency, IEEE Std 1003.1-2001 requires the latter behavior for all four commands. All four commands still consider it an error if the current line is at the beginning ( <control>-B, <control>-U) or end ( <control>-F, <control>-D) of the file. Historically, the <control>-B and <control>-F commands skip two lines in order to include overlapping lines when a single command is entered. This makes less sense in the presence of a count, as there will be, by definition, no overlapping lines. The actual calculation used by historical implementations of the vi editor for <control>-B was:

 

((current first line) - count x (window edit option)) +2

and for <control>-F was:

 

((current first line) + count x (window edit option)) -2

This calculation does not work well when intermixing commands with and without counts; for example, 3 <control>-F is not equivalent to entering the <control>-F command three times, and is not reversible by entering the <control>-B command three times. For consistency with other vi commands that take counts, IEEE Std 1003.1-2001 requires a different calculation.  

Scroll Forward

The 4BSD and System V implementations of vi differed on the initial value used by the scroll command. 4BSD used:

 

((window edit option) +1) /2

while System V used the value of the scroll edit option. The System V version is specified by IEEE Std 1003.1-2001 because the standard developers believed that it was more intuitive and permitted the user a method of setting the scroll value initially without also setting the number of lines that are displayed.  

Scroll Forward by Line

Historically, the <control>-E and <control>-Y commands considered it an error if the last and first lines, respectively, were already on the screen. IEEE Std 1003.1-2001 requires conformance to historical practice. Historically, the <control>-E and <control>-Y commands had no effect in open mode. For simplicity and consistency of specification, IEEE Std 1003.1-2001 requires that they behave as usual, albeit with a single line screen.  

Clear and Redisplay

The historical <control>-L command refreshed the screen exactly as it was supposed to be currently displayed, replacing any '@' characters for lines that had been deleted but not updated on the screen with refreshed '@' characters. The intent of the <control>-L command is to refresh when the screen has been accidentally overwritten; for example, by a write command from another user, or modem noise.  

Redraw Screen

The historical <control>-R command redisplayed only when necessary to update lines that had been deleted but not updated on the screen and that were flagged with '@' characters. There is no requirement that the screen be in any way refreshed if no lines of this form are currently displayed. IEEE Std 1003.1-2001 permits implementations to extend this command to refresh lines on the screen flagged with '@' characters because they are too long to be displayed in the current framework; however, the current line and column need not be modified.  

Search for tagstring

Historically, the first non- <blank> at or after the cursor was the first character, and all subsequent characters that were word characters, up to the end of the line, were included. For example, with the cursor on the leading space or on the '#' character in the text "#bar@" , the tag was "#bar" . On the character 'b' it was "bar" , and on the 'a' it was "ar" . IEEE Std 1003.1-2001 requires this behavior.  

Replace Text with Results from Shell Command

Historically, the <, >, and ! commands considered most cursor motions other than line-oriented motions an error; for example, the command >/foo<CR> succeeded, while the command >l failed, even though the text region described by the two commands might be identical. For consistency, all three commands only consider entire lines and not partial lines, and the region is defined as any line that contains a character that was specified by the motion.  

Move to Matching Character

Other matching characters have been left implementation-defined in order to allow extensions such as matching '<' and '>' for searching HTML, or #ifdef, #else, and #endif for searching C source.  

Repeat Substitution

IEEE Std 1003.1-2001 requires that any c and g flags specified to the previous substitute command be ignored; however, the r flag may still apply, if supported by the implementation.  

Return to Previous (Context or Section)

The [[, ]], (, ), {, and } commands are all affected by "section boundaries", but in some historical implementations not all of the commands recognize the same section boundaries. This is a bug, not a feature, and a unique section-boundary algorithm was not described for each command. One special case that is preserved is that the sentence command moves to the end of the last line of the edit buffer while the other commands go to the beginning, in order to preserve the traditional character cut semantics of the sentence command. Historically, vi section boundaries at the beginning and end of the edit buffer were the first non- <blank> on the first and last lines of the edit buffer if one exists; otherwise, the last character of the first and last lines of the edit buffer if one exists. To increase consistency with other section locations, this has been simplified by IEEE Std 1003.1-2001 to the first character of the first and last lines of the edit buffer, or the first and the last lines of the edit buffer if they are empty.

Sentence boundaries were problematic in the historical vi. They were not only the boundaries as defined for the section and paragraph commands, but they were the first non- <blank> that occurred after those boundaries, as well. Historically, the vi section commands were documented as taking an optional window size as a count preceding the command. This was not implemented in historical versions, so IEEE Std 1003.1-2001 requires that the count repeat the command, for consistency with other vi commands.  

Repeat

Historically, mapped commands other than text input commands could not be repeated using the period command. IEEE Std 1003.1-2001 requires conformance to historical practice.

The restrictions on the interpretation of special characters (for example, <control>-H) in the repetition of text input mode commands is intended to match historical practice. For example, given the input sequence:

 

iab<control>-H<control>-H<control>-Hdef<escape>

the user should be informed of an error when the sequence is first entered, but not during a command repetition. The character <control>-T is specifically exempted from this restriction. Historical implementations of vi ignored <control>-T characters that were input in the original command during command repetition. IEEE Std 1003.1-2001 prohibits this behavior.  

Find Regular Expression

Historically, commands did not affect the line searched to or from if the motion command was a search ( /, ?, N, n) and the final position was the start/end of the line. There were some special cases and vi was not consistent. IEEE Std 1003.1-2001 does not permit this behavior, for consistency. Historical implementations permitted but were unable to handle searches as motion commands that wrapped (that is, due to the edit option wrapscan) to the original location. IEEE Std 1003.1-2001 requires that this behavior be treated as an error.

Historically, the syntax "/RE/0" was used to force the command to cut text in line mode. IEEE Std 1003.1-2001 requires conformance to historical practice.

Historically, in open mode, a z specified to a search command redisplayed the current line instead of displaying the current screen with the current line highlighted. For consistency and simplicity of specification, IEEE Std 1003.1-2001 does not permit this behavior.

Historically, trailing z commands were permitted and ignored if entered as part of a search used as a motion command. For consistency and simplicity of specification, IEEE Std 1003.1-2001 does not permit this behavior.  

Execute an ex Command

Historically, vi implementations restricted the commands that could be entered on the colon command line (for example, append and change), and some other commands were known to cause them to fail catastrophically. For consistency, IEEE Std 1003.1-2001 does not permit these restrictions. When executing an ex command by entering :, it is not possible to enter a <newline> as part of the command because it is considered the end of the command. A different approach is to enter ex command mode by using the vi Q command (and later resuming visual mode with the ex vi command). In ex command mode, the single-line limitation does not exist. So, for example, the following is valid:

 

Q
s/break here/break\
here/
vi

IEEE Std 1003.1-2001 requires that, if the ex command overwrites any part of the screen that would be erased by a refresh, vi pauses for a character from the user. Historically, this character could be any character; for example, a character input by the user before the message appeared, or even a mapped character. This is probably a bug, but implementations that have tried to be more rigorous by requiring that the user enter a specific character, or that the user enter a character after the message was displayed, have been forced by user indignation back into historical behavior. IEEE Std 1003.1-2001 requires conformance to historical practice.  

Shift Left (Right)

Refer to the Rationale for the ! and / commands. Historically, the < and > commands sometimes moved the cursor to the first non- <blank> (for example if the command was repeated or with _ as the motion command), and sometimes left it unchanged. IEEE Std 1003.1-2001 does not permit this inconsistency, requiring instead that the cursor always move to the first non- <blank>. Historically, the < and > commands did not support buffer arguments, although some implementations allow the specification of an optional buffer. This behavior is neither required nor disallowed by IEEE Std 1003.1-2001.  

Execute

Historically, buffers could execute other buffers, and loops, infinite and otherwise, were possible. IEEE Std 1003.1-2001 requires conformance to historical practice. The * buffer syntax of ex is not required in vi, because it is not historical practice and has been used in some vi implementations to support additional scripting languages.  

Reverse Case

Historically, the ~ command ignored any associated count, and acted only on the characters in the current line. For consistency with other vi commands, IEEE Std 1003.1-2001 requires that an associated count act on the next count characters, and that the command move to subsequent lines if warranted by count, to make it possible to modify large pieces of text in a reasonably efficient manner. There exist vi implementations that optionally require an associated motion command for the ~ command. Implementations supporting this functionality are encouraged to base it on the tildedop edit option and handle the text regions and cursor positioning identically to the yank command.  

Append

Historically, counts specified to the A, a, I, and i commands repeated the input of the first line count times, and did not repeat the subsequent lines of the input text. IEEE Std 1003.1-2001 requires that the entire text input be repeated count times.  

Move Backward to Preceding Word

Historically, vi became confused if word commands were used as motion commands in empty files. IEEE Std 1003.1-2001 requires that this be an error. Historical implementations of vi had a large number of bugs in the word movement commands, and they varied greatly in behavior in the presence of empty lines, "words" made up of a single character, and lines containing only <blank>s. For consistency and simplicity of specification, IEEE Std 1003.1-2001 does not permit this behavior.  

Change to End-of-Line

Some historical implementations of the C command did not behave as described by IEEE Std 1003.1-2001 when the $ key was remapped because they were implemented by pushing the $ key onto the input queue and reprocessing it. IEEE Std 1003.1-2001 does not permit this behavior. Historically, the C, S, and s commands did not copy replaced text into the numeric buffers. For consistency and simplicity of specification, IEEE Std 1003.1-2001 requires that they behave like their respective c commands in all respects.  

Delete

Historically, lines in open mode that were deleted were scrolled up, and an @ glyph written over the beginning of the line. In the case of terminals that are incapable of the necessary cursor motions, the editor erased the deleted line from the screen. IEEE Std 1003.1-2001 requires conformance to historical practice; that is, if the terminal cannot display the '@' character, the line cannot remain on the screen.  

Delete to End-of-Line

Some historical implementations of the D command did not behave as described by IEEE Std 1003.1-2001 when the $ key was remapped because they were implemented by pushing the $ key onto the input queue and reprocessing it. IEEE Std 1003.1-2001 does not permit this behavior.  

Join

An historical oddity of vi is that the commands J, 1J, and 2J are all equivalent. IEEE Std 1003.1-2001 requires conformance to historical practice. The vi J command is specified in terms of the ex join command with an ex command count value. The address correction for a count that is past the end of the edit buffer is necessary for historical compatibility for both ex and vi.  

Mark Position

Historical practice is that only lowercase letters, plus '`' and '" , could be used to mark a cursor position. IEEE Std 1003.1-2001 requires conformance to historical practice, but encourages implementations to support other characters as marks as well.  

Repeat Regular Expression Find (Forward and Reverse)

Historically, the N and n commands could not be used as motion components for the c command. With the exception of the cN command, which worked if the search crossed a line boundary, the text region would be discarded, and the user would not be in text input mode. For consistency and simplicity of specification, IEEE Std 1003.1-2001 does not permit this behavior.  

Insert Empty Line (Below and Above)

Historically, counts to the O and o commands were used as the number of physical lines to open, if the terminal was dumb and the slowopen option was not set. This was intended to minimize traffic over slow connections and repainting for dumb terminals. IEEE Std 1003.1-2001 does not permit this behavior, requiring that a count to the open command behave as for other text input commands. This change to historical practice was made for consistency, and because a superset of the functionality is provided by the slowopen edit option.  

Put from Buffer (Following and Before)

Historically, counts to the p and P commands were ignored if the buffer was a line mode buffer, but were (mostly) implemented as described in IEEE Std 1003.1-2001 if the buffer was a character mode buffer. Because implementations exist that do not have this limitation, and because pasting lines multiple times is generally useful, IEEE Std 1003.1-2001 requires that count be supported for all p and P commands.

Historical implementations of vi were widely known to have major problems in the p and P commands, particularly when unusual regions of text were copied into the edit buffer. The standard developers viewed these as bugs, and they are not permitted for consistency and simplicity of specification.

Historically, a P or p command (or an ex put command executed from open or visual mode) executed in an empty file, left an empty line as the first line of the file. For consistency and simplicity of specification, IEEE Std 1003.1-2001 does not permit this behavior.  

Replace Character

Historically, the r command did not correctly handle the erase and word erase characters as arguments, nor did it handle an associated count greater than 1 with a <carriage-return> argument, for which it replaced count characters with a single <newline>. IEEE Std 1003.1-2001 does not permit these inconsistencies.

Historically, the r command permitted the <control>-V escaping of entered characters, such as <ESC> and the <carriage-return>; however, it required two leading <control>-V characters instead of one. IEEE Std 1003.1-2001 requires that this be changed for consistency with the other text input commands of vi.

Historically, it is an error to enter the r command if there are less than count characters at or after the cursor in the line. While a reasonable and unambiguous extension would be to permit the r command on empty lines, it would require that too large a count be adjusted to match the number of characters at or after the cursor for consistency, which is sufficiently different from historical practice to be avoided. IEEE Std 1003.1-2001 requires conformance to historical practice.  

Replace Characters

Historically, if there were autoindent characters in the line on which the R command was run, and autoindent was set, the first <newline> would be properly indented and no characters would be replaced by the <newline>. Each additional <newline> would replace n characters, where n was the number of characters that were needed to indent the rest of the line to the proper indentation level. This behavior is a bug and is not permitted by IEEE Std 1003.1-2001.  

Undo

Historical practice for cursor positioning after undoing commands was mixed. In most cases, when undoing commands that affected a single line, the cursor was moved to the start of added or changed text, or immediately after deleted text. However, if the user had moved from the line being changed, the column was either set to the first non- <blank>, returned to the origin of the command, or remained unchanged. When undoing commands that affected multiple lines or entire lines, the cursor was moved to the first character in the first line restored. As an example of how inconsistent this was, a search, followed by an o text input command, followed by an undo would return the cursor to the location where the o command was entered, but a cw command followed by an o command followed by an undo would return the cursor to the first non- <blank> of the line. IEEE Std 1003.1-2001 requires the most useful of these behaviors, and discards the least useful, in the interest of consistency and simplicity of specification.  

Yank

Historically, the yank command did not move to the end of the motion if the motion was in the forward direction. It moved to the end of the motion if the motion was in the backward direction, except for the _ command, or for the G and ' commands when the end of the motion was on the current line. This was further complicated by the fact that for a number of motion commands, the yank command moved the cursor but did not update the screen; for example, a subsequent command would move the cursor from the end of the motion, even though the cursor on the screen had not reflected the cursor movement for the yank command. IEEE Std 1003.1-2001 requires that all yank commands associated with backward motions move the cursor to the end of the motion for consistency, and specifically, to make ' commands as motions consistent with search patterns as motions.  

Yank Current Line

Some historical implementations of the Y command did not behave as described by IEEE Std 1003.1-2001 when the '_' key was remapped because they were implemented by pushing the '_' key onto the input queue and reprocessing it. IEEE Std 1003.1-2001 does not permit this behavior.  

Redraw Window

Historically, the z command always redrew the screen. This is permitted but not required by IEEE Std 1003.1-2001, because of the frequent use of the z command in macros such as map n nz. for screen positioning, instead of its use to change the screen size. The standard developers believed that expanding or scrolling the screen offered a better interface for users. The ability to redraw the screen is preserved if the optional new window size is specified, and in the <control>-L and <control>-R commands.

The semantics of z^ are confusing at best. Historical practice is that the screen before the screen that ended with the specified line is displayed. IEEE Std 1003.1-2001 requires conformance to historical practice.

Historically, the z command would not display a partial line at the top or bottom of the screen. If the partial line would normally have been displayed at the bottom of the screen, the command worked, but the partial line was replaced with '@' characters. If the partial line would normally have been displayed at the top of the screen, the command would fail. For consistency and simplicity of specification, IEEE Std 1003.1-2001 does not permit this behavior.

Historically, the z command with a line specification of 1 ignored the command. For consistency and simplicity of specification, IEEE Std 1003.1-2001 does not permit this behavior.

Historically, the z command did not set the cursor column to the first non- <blank> for the character if the first screen was to be displayed, and was already displayed. For consistency and simplicity of specification, IEEE Std 1003.1-2001 does not permit this behavior.  

Input Mode Commands in vi

Historical implementations of vi did not permit the user to erase more than a single line of input, or to use normal erase characters such as line erase, worderase, and erase to erase autoindent characters. As there exist implementations of vi that do not have these limitations, both behaviors are permitted, but only historical practice is required. In the case of these extensions, vi is required to pause at the autoindent and previous line boundaries.

Historical implementations of vi updated only the portion of the screen where the current cursor character was displayed. For example, consider the vi input keystrokes:

 

iabcd<escape>0C<tab>

Historically, the <tab> would overwrite the characters "abcd" when it was displayed. Other implementations replace only the 'a' character with the <tab>, and then push the rest of the characters ahead of the cursor. Both implementations have problems. The historical implementation is probably visually nicer for the above example; however, for the keystrokes:

 

iabcd<ESC>0R<tab><ESC>

the historical implementation results in the string "bcd" disappearing and then magically reappearing when the <ESC> character is entered. IEEE Std 1003.1-2001 requires the former behavior when overwriting erase-columns-that is, overwriting characters that are no longer logically part of the edit buffer-and the latter behavior otherwise.

Historical implementations of vi discarded the <control>-D and <control>-T characters when they were entered at places where their command functionality was not appropriate. IEEE Std 1003.1-2001 requires that the <control>-T functionality always be available, and that <control>-D be treated as any other key when not operating on autoindent characters.  

NUL

Some historical implementations of vi limited the number of characters entered using the NUL input character to 256 bytes. IEEE Std 1003.1-2001 permits this limitation; however, implementations are encouraged to remove this limit.  

<control>-D

See also Rationale for the input mode command <newline>. The hidden assumptions in the <control>-D command (and in the vi autoindent specification in general) is that <space>s take up a single column on the screen and that <tab>s are comprised of an integral number of <space>s.  

<newline>

Implementations are permitted to rewrite autoindent characters in the line when <newline>, <carriage-return>, <control>-D, and <control>-T are entered, or when the shift commands are used, because historical implementations have both done so and found it necessary to do so. For example, a <control>-D when the cursor is preceded by a single <tab>, with tabstop set to 8, and shiftwidth set to 3, will result in the <tab> being replaced by several <space>s.  

<control>-T

See also the Rationale for the input mode command <newline>. Historically, <control>-T only worked if no non- <blank>s had yet been input in the current input line. In addition, the characters inserted by <control>-T were treated as autoindent characters, and could not be erased using normal user erase characters. Because implementations exist that do not have these limitations, and as moving to a column boundary is generally useful, IEEE Std 1003.1-2001 requires that both limitations be removed.  

<control>-V

Historically, vi used ^V, regardless of the value of the literal-next character of the terminal. IEEE Std 1003.1-2001 requires conformance to historical practice.

The uses described for <control>-V can also be accomplished with <control>-Q, which is useful on terminals that use <control>-V for the down-arrow function. However, most historical implementations use <control>-Q for the termios START character, so the editor will generally not receive the <control>-Q unless stty ixon mode is set to off. (In addition, some historical implementations of vi explicitly set ixon mode to on, so it was difficult for the user to set it to off.) Any of the command characters described in IEEE Std 1003.1-2001 can be made ineffective by their selection as termios control characters, using the stty utility or other methods described in the System Interfaces volume of IEEE Std 1003.1-2001.  

<ESC>

Historically, SIGINT alerted the terminal when used to end input mode. This behavior is permitted, but not required, by IEEE Std 1003.1-2001.  

FUTURE DIRECTIONS

None.  

SEE ALSO

ed , ex , stty  

COPYRIGHT

Portions of this text are reprinted and reproduced in electronic form from IEEE Std 1003.1, 2003 Edition, Standard for Information Technology -- Portable Operating System Interface (POSIX), The Open Group Base Specifications Issue 6, Copyright (C) 2001-2003 by the Institute of Electrical and Electronics Engineers, Inc and The Open Group. In the event of any discrepancy between this version and the original IEEE and The Open Group Standard, the original IEEE and The Open Group Standard is the referee document. The original Standard can be obtained online at http://www.opengroup.org/unix/online.html .

 


 

Index

PROLOG
NAME
SYNOPSIS
DESCRIPTION
OPTIONS
OPERANDS
STDIN
INPUT FILES
ENVIRONMENT VARIABLES
ASYNCHRONOUS EVENTS
STDOUT
STDERR
OUTPUT FILES
EXTENDED DESCRIPTION
Initialization in ex and vi
Command Descriptions in vi
Page Backwards
Scroll Forward
Scroll Forward by Line
Page Forward
Display Information
Move Cursor Backwards
Move Down
Clear and Redisplay
Move Up
Redraw Screen
Scroll Backward
Scroll Backward by Line
Edit the Alternate File
Terminate Command or Input Mode
Search for tagstring
Move Cursor Forward
Replace Text with Results from Shell Command
Move Cursor to End-of-Line
Move to Matching Character
Repeat Substitution
Return to Previous Context at Beginning of Line
Return to Previous Context
Return to Previous Section
Move to Next Section
Move to First Non-<blank> Position on Current Line
Current and Line Above
Move Back to Beginning of Sentence
Move Forward to Beginning of Sentence
Move Back to Preceding Paragraph
Move Forward to Next Paragraph
Move to Specific Column Position
Reverse Find Character
Repeat
Find Regular Expression
Move to First Character in Line
Execute an ex Command
Repeat Find
Shift Left
Shift Right
Scan Backwards for Regular Expression
Execute
Reverse Case
Append
Append at End-of-Line
Move Backward to Preceding Word
Move Backward to Preceding Bigword
Change
Change to End-of-Line
Delete
Delete to End-of-Line
Move to End-of-Word
Move to End-of-Bigword
Find Character in Current Line (Forward)
Find Character in Current Line (Reverse)
Move to Line
Move to Top of Screen
Insert Before Cursor
Insert at Beginning of Line
Join
Move to Bottom of Screen
Mark Position
Move to Middle of Screen
Repeat Regular Expression Find (Forward)
Repeat Regular Expression Find (Reverse)
Insert Empty Line Below
Insert Empty Line Above
Put from Buffer Following
Put from Buffer Before
Enter ex Mode
Replace Character
Replace Characters
Substitute Character
Substitute Lines
Move Cursor to Before Character (Forward)
Move Cursor to After Character (Reverse)
Undo
Undo Current Line
Move to Beginning of Word
Move to Beginning of Bigword
Delete Character at Cursor
Delete Character Before Cursor
Yank
Yank Current Line
Redraw Window
Exit
Input Mode Commands in vi
NUL
<control>-D
<control>-H
<newline>
<control>-T
<control>-U
<control>-V
<control>-W
<ESC>
EXIT STATUS
CONSEQUENCES OF ERRORS
APPLICATION USAGE
EXAMPLES
RATIONALE
Initialization in ex and vi
Command Descriptions in vi
Page Backwards
Scroll Forward
Scroll Forward by Line
Clear and Redisplay
Redraw Screen
Search for tagstring
Replace Text with Results from Shell Command
Move to Matching Character
Repeat Substitution
Return to Previous (Context or Section)
Repeat
Find Regular Expression
Execute an ex Command
Shift Left (Right)
Execute
Reverse Case
Append
Move Backward to Preceding Word
Change to End-of-Line
Delete
Delete to End-of-Line
Join
Mark Position
Repeat Regular Expression Find (Forward and Reverse)
Insert Empty Line (Below and Above)
Put from Buffer (Following and Before)
Replace Character
Replace Characters
Undo
Yank
Yank Current Line
Redraw Window
Input Mode Commands in vi
NUL
<control>-D
<newline>
<control>-T
<control>-V
<ESC>
FUTURE DIRECTIONS
SEE ALSO
COPYRIGHT

Author: Chris Burton
Last update: 2014-12-16


Manual: ps

 

PROLOG

This manual page is part of the POSIX Programmer's Manual. The Linux implementation of this interface may differ (consult the corresponding Linux manual page for details of Linux behavior), or the interface may not be implemented on Linux.  

NAME

ps - report process status  

SYNOPSIS

ps [-aA][-defl][-G grouplist][-o format]...[-p proclist][-t termlist]

 

[-U userlist][-g grouplist][-n namelist][-u userlist] 
 

DESCRIPTION

The ps utility shall write information about processes, subject to having the appropriate privileges to obtain information about those processes.

By default, ps shall select all processes with the same effective user ID as the current user and the same controlling terminal as the invoker.  

OPTIONS

The ps utility shall conform to the Base Definitions volume of IEEE Std 1003.1-2001, Section 12.2, Utility Syntax Guidelines.

The following options shall be supported:

-a
Write information for all processes associated with terminals. Implementations may omit session leaders from this list.
-A
Write information for all processes.
-d
Write information for all processes, except session leaders.
-e
Write information for all processes. (Equivalent to -A.)
-f
Generate a full listing. (See the STDOUT section for the contents of a full listing.)
-g  grouplist
Write information for processes whose session leaders are given in grouplist. The application shall ensure that the grouplist is a single argument in the form of a <blank> or comma-separated list.
-G  grouplist
Write information for processes whose real group ID numbers are given in grouplist. The application shall ensure that the grouplist is a single argument in the form of a <blank> or comma-separated list.
-l
Generate a long listing. (See STDOUT for the contents of a long listing.)
-n  namelist
Specify the name of an alternative system namelist file in place of the default. The name of the default file and the format of a namelist file are unspecified.
-o  format
Write information according to the format specification given in format. This is fully described in the STDOUT section. Multiple -o options can be specified; the format specification shall be interpreted as the <space>-separated concatenation of all the format option-arguments.
-p  proclist
Write information for processes whose process ID numbers are given in proclist. The application shall ensure that the proclist is a single argument in the form of a <blank> or comma-separated list.
-t  termlist
Write information for processes associated with terminals given in termlist. The application shall ensure that the termlist is a single argument in the form of a <blank> or comma-separated list. Terminal identifiers shall be given in an implementation-defined format.  On XSI-conformant systems, they shall be given in one of two forms: the device's filename (for example, tty04) or, if the device's filename starts with tty, just the identifier following the characters tty (for example, "04" ).
-u  userlist
Write information for processes whose user ID numbers or login names are given in userlist. The application shall ensure that the userlist is a single argument in the form of a <blank> or comma-separated list. In the listing, the numerical user ID shall be written unless the -f option is used, in which case the login name shall be written.
-U  userlist
Write information for processes whose real user ID numbers or login names are given in userlist. The application shall ensure that the userlist is a single argument in the form of a <blank> or comma-separated list.

 

With the exception of -o format, all of the options shown are used to select processes. If any are specified, the default list shall be ignored and ps shall select the processes represented by the inclusive OR of all the selection-criteria options.  

OPERANDS

None.  

STDIN

Not used.  

INPUT FILES

None.  

ENVIRONMENT VARIABLES

The following environment variables shall affect the execution of ps:

COLUMNS
Override the system-selected horizontal display line size, used to determine the number of text columns to display. See the Base Definitions volume of IEEE Std 1003.1-2001, Chapter 8, Environment Variables for valid values and results when it is unset or null.
LANG
Provide a default value for the internationalization variables that are unset or null. (See the Base Definitions volume of IEEE Std 1003.1-2001, Section 8.2, Internationalization Variables for the precedence of internationalization variables used to determine the values of locale categories.)
LC_ALL
If set to a non-empty string value, override the values of all the other internationalization variables.
LC_CTYPE
Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multi-byte characters in arguments).
LC_MESSAGES
Determine the locale that should be used to affect the format and contents of diagnostic messages written to standard error and informative messages written to standard output.
LC_TIME
Determine the format and contents of the date and time strings displayed.
NLSPATH
Determine the location of message catalogs for the processing of LC_MESSAGES .
TZ
Determine the timezone used to calculate date and time strings displayed. If TZ is unset or null, an unspecified default timezone shall be used.

 

 

ASYNCHRONOUS EVENTS

Default.  

STDOUT

When the -o option is not specified, the standard output format is unspecified.

On XSI-conformant systems, the output format shall be as follows. The column headings and descriptions of the columns in a ps listing are given below. The precise meanings of these fields are implementation-defined. The letters 'f' and 'l' (below) indicate the option ( full or long) that shall cause the corresponding heading to appear; all means that the heading always appears. Note that these two options determine only what information is provided for a process; they do not determine which processes are listed.

F (l)     Flags (octal and additive) associated with the process.      
S (l)     The state of the process.      
UID (f,l)     The user ID number of the process owner; the login name is printed under the -f option.      
PID (all)     The process ID of the process; it is possible to kill a process if this datum is known.      
PPID (f,l)     The process ID of the parent process.      
C (f,l)     Processor utilization for scheduling.      
PRI (l)     The priority of the process; higher numbers mean lower priority.      
NI (l)     Nice value; used in priority computation.      
ADDR (l)     The address of the process.      
SZ (l)     The size in blocks of the core image of the process.      
WCHAN (l)     The event for which the process is waiting or sleeping; if blank, the process is running.      
STIME (f)     Starting time of the process.      
TTY (all)     The controlling terminal for the process.      
TIME (all)     The cumulative execution time for the process.      
CMD (all)     The command name; the full command name and its arguments are written under the -f option.      

A process that has exited and has a parent, but has not yet been waited for by the parent, shall be marked defunct.

Under the option -f, ps tries to determine the command name and arguments given when the process was created by examining memory or the swap area. Failing this, the command name, as it would appear without the option -f, is written in square brackets.

The -o option allows the output format to be specified under user control.

The application shall ensure that the format specification is a list of names presented as a single argument, <blank> or comma-separated. Each variable has a default header. The default header can be overridden by appending an equals sign and the new text of the header. The rest of the characters in the argument shall be used as the header text. The fields specified shall be written in the order specified on the command line, and should be arranged in columns in the output. The field widths shall be selected by the system to be at least as wide as the header text (default or overridden value). If the header text is null, such as -o user=, the field width shall be at least as wide as the default header text. If all header text fields are null, no header line shall be written.

The following names are recognized in the POSIX locale:

ruser
The real user ID of the process. This shall be the textual user ID, if it can be obtained and the field width permits, or a decimal representation otherwise.
user
The effective user ID of the process. This shall be the textual user ID, if it can be obtained and the field width permits, or a decimal representation otherwise.
rgroup
The real group ID of the process. This shall be the textual group ID, if it can be obtained and the field width permits, or a decimal representation otherwise.
group
The effective group ID of the process. This shall be the textual group ID, if it can be obtained and the field width permits, or a decimal representation otherwise.
pid
The decimal value of the process ID.
ppid
The decimal value of the parent process ID.
pgid
The decimal value of the process group ID.
pcpu
The ratio of CPU time used recently to CPU time available in the same period, expressed as a percentage. The meaning of "recently" in this context is unspecified. The CPU time available is determined in an unspecified manner.
vsz
The size of the process in (virtual) memory in 1024 byte units as a decimal integer.
nice
The decimal value of the nice value of the process; see nice() .
etime
In the POSIX locale, the elapsed time since the process was started, in the form:

 

[[dd-]hh:]mm:ss

where dd shall represent the number of days, hh the number of hours, mm the number of minutes, and ss the number of seconds. The dd field shall be a decimal integer. The hh, mm, and ss fields shall be two-digit decimal integers padded on the left with zeros.

time
In the POSIX locale, the cumulative CPU time of the process in the form:

 

[dd-]hh:mm:ss

The dd, hh, mm, and ss fields shall be as described in the etime specifier.

tty
The name of the controlling terminal of the process (if any) in the same format used by the who utility.
comm
The name of the command being executed ( argv[0] value) as a string.
args
The command with all its arguments as a string. The implementation may truncate this value to the field width; it is implementation-defined whether any further truncation occurs. It is unspecified whether the string represented is a version of the argument list as it was passed to the command when it started, or is a version of the arguments as they may have been modified by the application. Applications cannot depend on being able to modify their argument list and having that modification be reflected in the output of ps.

 

Any field need not be meaningful in all implementations. In such a case a hyphen ( '-' ) should be output in place of the field value.

Only comm and args shall be allowed to contain <blank>s; all others shall not. Any implementation-defined variables shall be specified in the system documentation along with the default header and indicating whether the field may contain <blank>s.

The following table specifies the default header to be used in the POSIX locale corresponding to each format specifier.

 

Table: Variable Names and Default Headers in ps

Format Specifier Default Header     Format Specifier Default Header       
args COMMAND     ppid PPID       
comm COMMAND     rgroup RGROUP       
etime ELAPSED     ruser RUSER       
group GROUP     time TIME       
nice NI     tty TT       
pcpu %CPU     user USER       
pgid PGID     vsz VSZ       
pid PID               

 

STDERR

The standard error shall be used only for diagnostic messages.  

OUTPUT FILES

None.  

EXTENDED DESCRIPTION

None.  

EXIT STATUS

The following exit values shall be returned:

 0
Successful completion.
>0
An error occurred.

 

 

CONSEQUENCES OF ERRORS

Default.

The following sections are informative.  

APPLICATION USAGE

Things can change while ps is running; the snapshot it gives is only true for an instant, and might not be accurate by the time it is displayed.

The args format specifier is allowed to produce a truncated version of the command arguments. In some implementations, this information is no longer available when the ps utility is executed.

If the field width is too narrow to display a textual ID, the system may use a numeric version. Normally, the system would be expected to choose large enough field widths, but if a large number of fields were selected to write, it might squeeze fields to their minimum sizes to fit on one line. One way to ensure adequate width for the textual IDs is to override the default header for a field to make it larger than most or all user or group names.

There is no special quoting mechanism for header text. The header text is the rest of the argument. If multiple header changes are needed, multiple -o options can be used, such as:

 

ps -o "user=User Name" -o pid=Process\ ID

On some implementations, especially multi-level secure systems, ps may be severely restricted and produce information only about child processes owned by the user.  

EXAMPLES

The command:

 

ps -o user,pid,ppid=MOM -o args

writes at least the following in the POSIX locale:

 

 USER   PID   MOM   COMMAND
helene    34    12   ps -o uid,pid,ppid=MOM -o args

The contents of the COMMAND field need not be the same in all implementations, due to possible truncation.  

RATIONALE

There is very little commonality between BSD and System V implementations of ps. Many options conflict or have subtly different usages. The standard developers attempted to select a set of options for the base standard that were useful on a wide range of systems and selected options that either can be implemented on both BSD and System V-based systems without breaking the current implementations or where the options are sufficiently similar that any changes would not be unduly problematic for users or implementors.

It is recognized that on some implementations, especially multi-level secure systems, ps may be nearly useless. The default output has therefore been chosen such that it does not break historical implementations and also is likely to provide at least some useful information on most systems.

The major change is the addition of the format specification capability. The motivation for this invention is to provide a mechanism for users to access a wider range of system information, if the system permits it, in a portable manner. The fields chosen to appear in this volume of IEEE Std 1003.1-2001 were arrived at after considering what concepts were likely to be both reasonably useful to the "average" user and had a reasonable chance of being implemented on a wide range of systems. Again it is recognized that not all systems are able to provide all the information and, conversely, some may wish to provide more. It is hoped that the approach adopted will be sufficiently flexible and extensible to accommodate most systems. Implementations may be expected to introduce new format specifiers.

The default output should consist of a short listing containing the process ID, terminal name, cumulative execution time, and command name of each process.

The preference of the standard developers would have been to make the format specification an operand of the ps command. Unfortunately, BSD usage precluded this.

At one time a format was included to display the environment array of the process. This was deleted because there is no portable way to display it.

The -A option is equivalent to the BSD -g and the SVID -e. Because the two systems differed, a mnemonic compromise was selected.

The -a option is described with some optional behavior because the SVID omits session leaders, but BSD does not.

In an early proposal, format specifiers appeared for priority and start time. The former was not defined adequately in this volume of IEEE Std 1003.1-2001 and was removed in deference to the defined nice value; the latter because elapsed time was considered to be more useful.

In a new BSD version of ps, a -O option can be used to write all of the default information, followed by additional format specifiers. This was not adopted because the default output is implementation-defined. Nevertheless, this is a useful option that should be reserved for that purpose. In the -o option for the POSIX Shell and Utilities ps, the format is the concatenation of each -o. Therefore, the user can have an alias or function that defines the beginning of their desired format and add more fields to the end of the output in certain cases where that would be useful.

The format of the terminal name is unspecified, but the descriptions of ps, talk, who, and write require that they all use the same format.

The pcpu field indicates that the CPU time available is determined in an unspecified manner. This is because it is difficult to express an algorithm that is useful across all possible machine architectures. Historical counterparts to this value have attempted to show percentage of use in the recent past, such as the preceding minute. Frequently, these values for all processes did not add up to 100%. Implementations are encouraged to provide data in this field to users that will help them identify processes currently affecting the performance of the system.  

FUTURE DIRECTIONS

None.  

SEE ALSO

kill() , nice() , renice  

COPYRIGHT

Portions of this text are reprinted and reproduced in electronic form from IEEE Std 1003.1, 2003 Edition, Standard for Information Technology -- Portable Operating System Interface (POSIX), The Open Group Base Specifications Issue 6, Copyright (C) 2001-2003 by the Institute of Electrical and Electronics Engineers, Inc and The Open Group. In the event of any discrepancy between this version and the original IEEE and The Open Group Standard, the original IEEE and The Open Group Standard is the referee document. The original Standard can be obtained online at http://www.opengroup.org/unix/online.html .

 


 

Index

PROLOG
NAME
SYNOPSIS
DESCRIPTION
OPTIONS
OPERANDS
STDIN
INPUT FILES
ENVIRONMENT VARIABLES
ASYNCHRONOUS EVENTS
STDOUT
STDERR
OUTPUT FILES
EXTENDED DESCRIPTION
EXIT STATUS
CONSEQUENCES OF ERRORS
APPLICATION USAGE
EXAMPLES
RATIONALE
FUTURE DIRECTIONS
SEE ALSO
COPYRIGHT

Author: Chris Burton
Last update: 2014-12-16


Software » Microsoft Windows

Fixing a blank ipconfig output [Windows XP]

This came about after installing Panda Software Internet Security 2008 [12.00.00]

Resolution

  1. Click the Start button and select Run.
  2. Enter “cmd” and press “OK”.
  3. Type the command “regsvr32 netshell.dll” and press Enter.
  4. Type “exit” and press Enter to close the shell.
  5. Open “Network Connections”
  6. Right click on the Interface causing the problems and select “Disable”.
  7. Right click on the Interface causing the problems and select “Enable”.

The network interface should now be operational.

Author: Chris Burton
Last update: 2014-12-16


How do I disable Aero snap?

To disable Aero snap (Dragging windows off the edge get arranged into half of the screen):

Click on Start->Control Panel

Select the "Ease of Access center" category

Select  the option for option for "Change how your mouse works" under the "Ease of Access center" heading.

Towards the bottom of the page is the option "Prevent windows from being automatically arranged when moved to the edge of the screen" - Check this option and click "OK".

Now when you drag windows to the edge of the screen they will not be expanded to half screen size.

Author: Chris Burton
Last update: 2014-12-16


How do I clear a saved password for a network share?

To clear a saved password for a network share to allow you to use a new username/password use:

 

net use \\<sharename> /delete /yes

 

Replacing the <sharename> with the share you wish to clear the password for.

Author: Chris Burton
Last update: 2014-12-16


How do I activate Windows (w2k8/web/etc) using a MAK from the command line?

To change the licence key and activate Windows:

  1. Login to the server and run an Administrator command prompt.
  2. Type the following replacing the text with your MAK key to setup the key.
    slmgr.vbs /ipk <MAK>
     
  3. To activate windows run the following command
    slmgr.vbs /ato
     
  4. Windows should now be using the new MAK be activated over the internet.

Author: Chris Burton
Last update: 2014-12-16


Unable to login to Windows Virtual PC with error "Unable to log you on because of an account restriction"

This can be caused by the user account not having a password set.

This will only cause problems when "Integration Features" are enabled, to disable them click on the "Tools" menu and select "Disable Integration Features" which will then take you to the logged in machine. From here you can reset the password on the account and then re-enable "Integration Features" via the Tools menu and then login using the new password.

Author: Chris Burton
Last update: 2014-12-17


How do I open a new window in Google Chrome?

Press Ctrl-N to open the new window.

Author: Chris Burton
Last update: 2014-12-16


How do I open a new tab in Google Chrome?

Press Ctrl-T

Author: Chris Burton
Last update: 2014-12-16


google, chrome, new window, browser, shortcut, keyboard

Press Ctrl-Shift-N

Author: Chris Burton
Last update: 2014-12-16


How do I open a link in a new (background) tab in Google Chrome?

Using the mouse hold Ctrl key then press the left (primary) mouse button, or press the middle mouse button.

Author: Chris Burton
Last update: 2014-12-16


How do I open a link in a new (active) tab in Google Chrome?

Hold down Ctrl+Shift then press the left (primary) mouse button.

Author: Chris Burton
Last update: 2014-12-16


How do I open a link in a new window in Google Chrome?

Hold down Shift and then press the left (primary) mouse button.

Author: Chris Burton
Last update: 2014-12-16


How do I close the active tab in Google Chrome?

Press Ctrl-W

Author: Chris Burton
Last update: 2014-12-16


How to I switch between browser tabs in Google Chrome?

You can use Ctrl-PgUp and Ctrl-PgDn to move forwards and backwards in the tabs.

Alternatively you can move directly to any of the first 8 tabs using Ctrl-<tab number>

Author: Chris Burton
Last update: 2014-12-16


Unable to start Virtual PC "host processor type mismatch or lack of hardware-assisted virtualization support in the system".

To start your virtual machines when you recieve the error "<machine> could not be restored because of either host processor type mismatch or lack of hardware-assisted virtualization support in the system." you will need to remove the hybernation file.

 

1) Open Windows Explorer and browse to "C:\Users\<username>\AppData\Local\Microsoft\Windows Virtual PC\Virtual Machines" replaceing C: and <username> as required.

2) You should now see the .vsv files for the VM with the starting problem.

3) Remove (or rename) the vsv file for that VM.

4) You should now be able to start your VM normally.

Author: Chris Burton
Last update: 2014-12-17


How do I log into SmarterStats as the admin?

To login into Smarter Stats as the admin user you need to use:

 

Site ID: admin
Username: admin
Password: <your password>

You should now be logged into SmarterStats as the admin user. If you have changed the name of the admin user you will need to use this updated name whilst logging in.

Author: Chris Burton
Last update: 2014-12-16


DotNetPanel / WebSitePanel custom errors do not work

If IIS has been configured to show detailed errors by default custom error (404 etc) handlers will not function correctly, to resolve this edit your "web.config" file editing the "httpErrors" start tag to include the "errorMode" parameter as shown below (you will need to add the text into the existing tag) similar to below:

<httpErrors errorMode="DetailedLocalOnly">

When you view your website with this in the web.config file you should see custom errors displayed.

Author: Chris Burton
Last update: 2014-12-16


Software » Databases » MySQL

How do I fix Unable to connect to database: saved admin password is incorrect (Plesk)?

ERROR: PleskFatalException
Unable to connect to database: saved admin password is incorrect.

0: /usr/local/psa/admin/plib/common_func.php3:190
psaerror(string ‘Unable to connect to database: saved admin password is
incorrect.’)
1: /usr/local/psa/admin/auto_prepend/auth.php3:93

The following instructions assume your Plesk admin user is “admin” and you with to reset the password to “setup”. You can check the password Plesk is expecting to use by doing “cat /etc/psa/.psa.shadow”.

  1. Using your favoured editor create the file “/root/mysqlreset.sql” with the content “UPDATE mysql.user SET Password=PASSWORD(’setup’) WHERE User=’admin’;FLUSH PRIVILEGES;”.
  2. Stop the mysql server “/etc/init.d/mysqld stop” (this will cause issues with any sites utilising MySQL).
  3. Start the MySQL server using the initial SQL file created above “mysqld_safe –init-file=/root/mysqlreset.sql &”.
  4. This should have reset the password and you can now restart MySQL.

You should now be able to login to the Plesk control panel via http://<sitename>:8443/.

Author: Chris Burton
Last update: 2014-12-16


How do I graphing mysql statistics in Cacti?

Graphing mysql statistics in Cacti

The HOWTO below will guide you through creating statistics graphs for MySQL usage in Cacti.

 

We have modified the authentication to use the standard mtop user which was already on our system.

I saved this script as /usr/local/bin/cacti-mysqlstats.php and set it executable with:

chmod a+rx /usr/local/bin/cacti-mysqlstats.php

Here is a local cached copy of the script from: http://www.cacti.net/additional_scripts.php [Gets statistics from MySQL server]
#!/usr/bin/php -q
<?

/* sql_stat – Gets statistics from MySQL server

Takes one argument (Variable name) and returns it’s value.
Good replacement for the sql.php script.

some interesting variables:

Questions – queries executed since last restart.
Threads_running – running queries now
Threads_connected – connected clients
.
.
For more info RTFM or mail me:

Martin Spasov (mspasov@techno-link.com)
This script is distributed under GPL.

*/

$var = $argv[1];
mysql_connect(“localhost”, “mysqltop”) or die (“0″);
$res = mysql_query(“SHOW STATUS”) or die(“0″);
while ($a = mysql_fetch_row($res)) {
$stat[$a[0]] = $a[1];
}
printf(“%s”, $stat[$var]);
mysql_close();
?>

Data Input Method

To graph the data we need an input method

  1. Click – Data Input Methods (left hand menu)

To graph Queries

  1. Click – Add (top right hand corner)
  2. Should now show “Data Input Methods [new]“
  3. Set: Name -> Mysql – “Queries”
  4. Set: Input Type -> Query/Command
  5. Set: Input String -> “/usr/local/bin/cacti-mysqlstats.php Queries”
  6. Click – Create
  7. Click Output Fields -> Add
  8. Set: Field [Output] -> “queries”
  9. Set: Friendly Name -> “Queries”
  10. Click – Create
  11. Click – Save

Data Templates

  1. Click – Data Templates

To graph Queries

  1. Click – Add (top right hand corner)
  2. Set: Data Templates [new] -> “Mysql – Queries”
  3. Tick: Data Source “Use Per-Data Source Value (Ignore this Value)”
  4. Set: Data Input Method -> Mysql – Queries

Author: Chris Burton
Last update: 2014-12-16


Why is cPanel is showing disk usage when the account has no database?

Sometimes cPanel does not remove the database correctly and leaves behind files.

 

Take a backup and then remove the files in

 

/var/cpanel/datastore/<username>/

 

Then run

 

/scripts/fixmysql

 

You should now the correct information showing in the control panel.

Author: Chris Burton
Last update: 2014-12-16


What is the MySQL password for OnApp installation?

You can view the MySQL database name and password for an onapp in the following file.

cat /onapp/interface/config/database.yml

You can then use this information to connect to the OnApp MySQL database.

Author: Chris Burton
Last update: 2014-12-16


Software » Databases » PostgreSQL

How do I get a list of all databases on pgsql/PostgreSQL?

To get a list of all databases on PostgreSQL su to the postgres user:

 

su - postgres

 

Once you have switched to the postgres user simply issue the following command to list all databases.

 

psql -l

You should now see the full list of PostgreSQL databases on this system.

 

Author: Chris Burton
Last update: 2014-12-16


How do I backup a pgsql/PostgreSQL database?

To get a list of all databases on PostgreSQL su to the postgres user:

 

su - postgres

 

Once you have switched to the postgres user simply issue the following command to backup the database.

 

pg_dump <database> > <databasebackup.sql>

You should now have a complete backup of the database content.

Author: Chris Burton
Last update: 2014-12-16


Software » Databases » MongoDB

Mongodb - How do I view the current process/operation?

To view the current operation simply connect to mongo and issue the following command:

db.currentOp();

This will return a BSON object detailing the current operation.

Author: Chris Burton
Last update: 2014-12-16


How do I pause/unpause shard balancing in MongoDB?

During index creations and other long operations it is sometimes helpful to pause shard rebalancing.

To pause/disable sharding connect with the console and run

use config
db.settings.update( { _id: "balancer" }, { $set : { stopped: true } } , true );

To unpause/re-enable shard balancing connect to the console again and run

use config
db.settings.update( { _id: "balancer" }, { $set : { stopped: false } } , true );

Author: Chris Burton
Last update: 2014-12-16


Software » Microsoft Windows » Virtualisation

Unable to login to Windows Virtual PC with error "Unable to log you on because of an account restriction"

This can be caused by the user account not having a password set.

This will only cause problems when "Integration Features" are enabled, to disable them click on the "Tools" menu and select "Disable Integration Features" which will then take you to the logged in machine. From here you can reset the password on the account and then re-enable "Integration Features" via the Tools menu and then login using the new password.

Author: Chris Burton
Last update: 2014-12-17


Unable to start Virtual PC "host processor type mismatch or lack of hardware-assisted virtualization support in the system".

To start your virtual machines when you recieve the error "<machine> could not be restored because of either host processor type mismatch or lack of hardware-assisted virtualization support in the system." you will need to remove the hybernation file.

 

1) Open Windows Explorer and browse to "C:\Users\<username>\AppData\Local\Microsoft\Windows Virtual PC\Virtual Machines" replaceing C: and <username> as required.

2) You should now see the .vsv files for the VM with the starting problem.

3) Remove (or rename) the vsv file for that VM.

4) You should now be able to start your VM normally.

Author: Chris Burton
Last update: 2014-12-17


Software » Linux » Virtualisation

Citrix errror when exporting VM - Export failed due to a block checksum mismatch. Please retry the export

The error "Export failed due to a block checksum mismatch. Please retry the export." can normally be resolved by rebooting the Citrix VM host.

Author: Chris Burton
Last update: 2014-12-17


How do I setup CentOS 7 for KVM Virtualisation with a bridged network [OVH/Hetzner]?

Coming from a minimal install the first step is to install the required software

# yum install net-tools qemu-kvm libvirt virt-install bridge-utils bind-utils

Convert the network configuration to a static IP address

Both Hetzner and OVH supply their dedicated servers were the network is configured via DHCP, this needs to be updated to a static configuration for the bridged network for KVM to work correctly.

To view the current network configuration run.

# ifconfig

With CentOS 7 the days of ethX are gone (unless it's OVH), they will now be labelled something like enp6s0

enp6s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.10.10.10  netmask 255.255.255.224  broadcast 176.9.36.255
        inet6 fe80::8e89:a5ff:fe63:b8ef  prefixlen 64  scopeid 0x20<link>
        ether 8c:89:a5:63:b8:ef  txqueuelen 1000  (Ethernet)
        RX packets 69106191  bytes 7806556849 (7.2 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 197727632  bytes 274850510321 (255.9 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

You will also need the gateway to setup the network.

# ip route

The output should have line prefixed with "default via".

default via 10.10.10.1 dev enp6s0  proto static  metric 1024

In the example above the default gateway is 10.10.10.1.

Replacing the name of your network device create a backup of the current network configuration (this must be a PREFIX otherwise it will be treated as a network device and will be configured on boot causing issues).

# cp /etc/sysconfig/network-scripts/ifcfg-enp6s0 /etc/sysconfig/network-scripts/bak.ifcfg-enp6s0

 Edit the configuration file (again replacing your network device name as required).

# vi /etc/sysconfig/network-scripts/ifcfg-enp6s0

You should see something like the following

# Generated by dracut initrd
DEVICE="enp6s0"
ONBOOT=yes
NETBOOT=yes
UUID="d2183429-2b16-43b1-99d6-433cf3c386ab"
IPV6INIT=yes
BOOTPROTO=dhcp
HWADDR="8c:89:a5:61:a8:ef"
TYPE=Ethernet
NAME="enp6s0"
DNS1="213.133.99.99"

 This needs to be changed to look like the following (dns shown are for Hetzner).

DEVICE="enp6s0"
ONBOOT=yes
NETBOOT=no
NM_CONTROLLED="no"
BOOTPROTO=static
HWADDR="8c:89:a5:61:a8:ef"
TYPE=Ethernet
NAME="enp6s0"
DNS1="213.133.98.98"
DNS2="213.133.99.99"
DNS3="213.133.100.100"
IPADDR=10.10.10.10
NETMASK=255.255.255.224
GATEWAY=10.10.10.1

Replacing the IPADDR/NETMASK with the inet initially shown on the network device, the GATEWAY IP from the output of  the "ip route" command.

Initially I would try restarting the network without a reboot 

# service network restart

If the network has been configured correctly the command prompt should return. If the command prompt doesn't come back you will need to boot the server in recovery mode and then either revert the changes from the bak file or correct any errors.

I would also advise doing reboot to ensure the network will start up like this as this is the basis of the network used later.

# reboot

Again if the server comes back after the reboot all is good, otherwise you will need to boot into recovery and resolve the issue.

Setup libvirt and configure the bridge

Start and enable libvirtd to start at system boot.

# systemctl start libvirtd
# systemctl enable libvirtd

Again we now need to copy the network file (changing the filename to your device as required) to make setting up the bridge network configration a little easier.

# cp /etc/sysconfig/network-scripts/ifcfg-enp6s0 /etc/sysconfig/network-scripts/ifcfg-br0

Edit the config for the network device.

# vi /etc/sysconfig/network-scripts/ifcfg-enp6s0

Comment out the HWADDR, TYPE, NAME, DNS1/DNS2/DNS3, IPADDR, NETMASK and GATEWAY and add the BRIDGE line which should leave you with something like this (we are calling the bridge to use with KVM "br0").

DEVICE="enp6s0"
ONBOOT=yes
NETBOOT=no
NM_CONTROLLED="no"
#BOOTPROTO=static
#HWADDR="8c:89:a5:61:a8:ef"
#TYPE=Ethernet
#NAME="enp6s0"
#DNS1="213.133.98.98"
#DNS2="213.133.99.99"
#DNS3="213.133.100.100"
#IPADDR=10.10.10.10
#NETMASK=255.255.255.224
#GATEWAY=10.10.10.1
BRIDGE=br0

 And then edit the new file for the bridge configuration.

# vi /etc/sysconfig/network-scripts/ifcfg-br0

After updating the DEVICE, TYPE, NAME and comment out the GATEWAY you should have something like.

DEVICE="br0"
ONBOOT=yes
NM_CONTROLLED="no"
NETBOOT=no
IPV6INIT=yes
BOOTPROTO=static
#HWADDR="8c:89:a5:63:b8:ef"
TYPE=Bridge
NAME="br0"
DNS1="213.133.99.99"
IPADDR=10.10.10.10
NETMASK=255.255.255.224
#GATEWAY=10.10.10.1

You will now need to add the commented out GATEWAY line to your network configuration

# vi /etc/sysconfig/network

And add the removed GATEWAY line

GATEWAY=10.10.10.1

Once these fines have been updated and checked restart the network service

# service network restart

Now check your new bridge has the correct configuration with the following commans.

# ifconfig br0
# ifconfig enp6s0

The br0 device should now have the IP address and the enp6s0 device should have no IP address associated with it.

Once you are happy with the configuration reboot the server.

# reboot

If everything looks OK after reboot you can now proceed with your KVM system and setup new Virtual Servers (with virt-manager for example). If your server doesn't come back after a few minutes you will need to boot into recovery mode and check the configuration files.

Author: Chris Burton
Last update: 2018-05-05


How do I make a KVM Virtual Server start on reboot?

Run the following command replacing SERVER_NAME with the name of the VM.

virsh autostart SERVER_NAME

You should then be notified the Domain has been marked to auto start.

Author: Chris Burton
Last update: 2014-12-17


How do I use a static network device name for KVM (to graph in Cacti for example)?

To force a static (pre-defined name for a KVM Virtual Machine you will need to edit the configuration file for the VM (replace VM_NAME with the name of the domain).

vi /etc/libvirt/qemu/VM_NAME.xml

Find the section of the file starting with "<interface type='bridge'>" and then add the following (replacing DEV_NAME with the name of the network device to use on the host).

<target dev='DEV_NAME'/>

You will then need to bring in the new change and stop/start the VM for the changes to take effect.

virsh define VM_NAME.xml
virsh shutdown VM_NAME
virsh start VM_NAME

You should now be able to see the new network device name on the host by using "ifconfig". This new device can now be picked by you Cacti for example and used to graph the VM using a static network device name.

Author: Chris Burton
Last update: 2014-12-17


How do I configure KVM to suspend/restore virtual machines when the host is rebooted?

To suspend a VM on host shutdown you will need to configure the following file

vi /etc/sysconfig/libvirt-guests

Set the following options (which are located throughout the configuration file).

ON_BOOT=start
START_DELAY=5
ON_SHUTDOWN=suspend

You might not want to use START_DELAY=5, but I would rather the virtual machines start up with a little staggering rather than concurrently.

The libvirt-guests service also needs to be enabled and started.

systemctl start libvirt-guests
systemctl enable libvirt-guests

You will now be able to reboot the host machine to install updates etc. and the virtual machines will be suspended and restored rather than shutdown interrupting their jobs.

Author: Chris Burton
Last update: 2014-12-17


How do I rename a KVM VM?

This example works on CentOS 7, but will likely work on other distributions too.

 

Save a copy of the Virtual Machines configuration (replacing VMNAME with the name of your VM as shown with "virsh list --all".

virsh dumpxml VMNAME > VMNAME.xml

Edit the VMNAME.xml file and search for 

<name>VMNAME</name>

and replace this with the new name for the VM and save the file.

Shutdown the VM (if the VM doesn't shutdown you can run "virsh destroy VMNAME" although data may be lost).

virsh shutdown VMNAME

To swap the server we need to undefine the old VM and define the new VM.

virsh undefine VMNAME
virsh define NEW_VMNAME VMNAME.xml

You can now start the VM to complete the rename.

virsh start NEW_VMNAME

 

Author: Chris Burton
Last update: 2015-01-29


Hardware » Cluster HAT

How do I enable autologin on TTY running on the Gadget Mode serial port?

The Cluster HAT allows you to log into the Pi Zeros via the (Gadget USB) Serial port, sometimes it's helpful to skip typing in the password.

To enable auto logins on the TTY running on any serial port on the Pi Zeros log into each Pi Zero run the following command and reboot.

sudo sed -i "s#agetty --noclear#agetty --autologin pi --noclear #" /lib/systemd/system/getty@.service

And to disable it run the following command and reboot.

sudo sed -i "s#agetty --autologin pi --noclear#agetty --noclear #" /lib/systemd/system/getty@.service

To login automatically using a different username replace the "pi" username in the above commands.

When enabling this all users with access to the Controller Pi will be able to log into the Pi Zeros without any authentication.

Author: Chris Burton
Last update: 2016-10-14


How do I create and copy the SSH key from the Controller Pi to the Pi Zeros to allow passwordless logins?

Assuming you haven't already created a public/private key on the Controller Pi run the following.

ssh-keygen -t rsa

Newer Raspberry Pi OS installations allow you to use your own username so please replace the "pi" user in the following commands with your username.

Press enter to save the key at the standard location (/home/pi/.ssh/id_rsa), you can optionally enter a passphrase but you will need to enter this every time you access the key.

Copy your public SSH key to the Pi Zeros 

for I in 1 2 3 4; do echo "Copying to p$I.local";ssh-copy-id pi@p$I.local;done

For each Pi you will need to type "yes" to connect if you haven't already previously logged into the Zero from the Controller using SSH and then enter your password.

You will now be able to log into the Pi Zeros using SSH without any key

ssh pi@p1.local

To run a command on all Pi Zeros sequentially you can reuse a portion of the above command.

for I in 1 2 3 4; do echo -n "p$I:";ssh pi@p$I.local uptime;done

You should see something similar to the output below which shows how long each Pi Zero has been booted, number of users, etc.

p1: 21:43:32 up 24 min,  3 users,  load average: 0.01, 0.02, 0.00
p2: 21:43:33 up 6 min,  2 users,  load average: 0.07, 0.08, 0.05
p3: 21:43:33 up 5 min,  2 users,  load average: 0.00, 0.07, 0.04
p4: 21:43:34 up 4 min,  2 users,  load average: 0.02, 0.11, 0.06

Author: Chris Burton
Last update: 2022-06-27


[LEGACY] How do I setup usbboot (no SD cards in the Pi Zeros) for the Cluster HAT?

See the current documentation on booting without SD cards, this legacy information is kept here in case it helps anyone.

I have made an image available to boot the Pi Zeros using usbboot/rpiboot - see https://8086.support/content/23/88/en/guide-to-using-the-rpiboot-test-image-on-the-cluster-hat_zero-stem-or-just-a-usb-cable.html so most of this guide is no longer needed if you want to try it yourself.

 

The following is a first attempt rough guide to setting up the Cluster HAT + Pi Zeros to boot using usbboot meaning no SD card is needed in the Pi Zeros in the Cluster HAT.

It uses about 5.7GB of disk space for 4 NFS roots (~800MB each root) but I'd advise using at least a 16GB SD card to allow some room for the filesystems to grow.

There are issues with this setup :( - see the bottom of this guide for a list.

The USB path (1-1.2.4) assumes you're connecting the Cluster HAT using the top left USB connector (looking at the end with the connectors) if this isn't the case your path will be different.

 

 

How does it work?

After powering on the Pi Zeros (without SD cards) they boot up and show up as a device. 
This is then picked up by a modified rpiboot (see "Modifications to rpiboot" below) which sends over the required files for the /boot directory.
The root filesystem is then mounted using NFS from the controller over the Internal network.
On the Pi Zero the network device usb0.10 (VLAN 10) is bridged to eth0 on the controller and obtains an IP address using the DHCP server on the network (see "Network Configuration" below)

Modifications to rpiboot

I have modified rpiboot to support "overlay" files based on the USB device path, this allows individual config.txt/cmdline.txt/etc. files to be sent to specific Pi Zeros using a single rpiboot process.

The new mode is enabled using the "-o" command line option and for example when used with "-d /boot" would first try to load files from /boot/USBPATH/ and if not found fallback to /boot/ . When using the Cluster HAT the USBPATH for P1 is "1-1.2.4", P2 is "1-1.2.3", P3 is "1-1.2.2" and P4 is "1-1.2.1" so when P1 is booting and looking for the config.txt file it will try to send /boot/1-1.2.4/config.txt but if the file isn't found will send /boot/config.txt as normal. This allows the use of a custom config.txt/cmdline.txt for each Pi Zero.

The source for the hacked/modified rpiboot can be found on https://github.com/burtyb/usbboot

Network Configiration

The network now uses VLANs internally to separate the Internal and External networks.

Your local network doesn't need to support VLANs these are all handled within the Controller/Pi Zeros. With the configuration described here the Pi Zeros will still obtain an IP address from the DHCP server on your local network. 

The original br0 interface which previously bridged eth0 (Controller) and ethpiX (Pi Zero) is removed and replaced by the Internal "brint" and External "brext" interfaces.

brint [Controller] is configured with the static IP address 172.19.180.254 and is bridged to the (untagged) ethpiX interface (usb0 on the Pi Zero size).

brext [Controller] is configured to obtain an IP address from the DHCP server on the local network (as br0 previously) and is bridged to eth0 on the Controller and the VLAN 10 interface ethpiX.10.

usb0 [Pi Zero] the untagged interface is used for NFSROOT (as initrd doesn't support VLAN). A static IP address 172.19.180.X is set for the Pi (so 172.19.180.1 for P1, ..180.2 for P2, etc). This is used for the NFSROOT but you can also use it for SSH/etc. access.

usb0.10 [Pi Zero] VLAN 10 is configured by dhcp (over brext to the eth0 interface on the controller) and will pickup an IP from the local network.

If you're using multiple Cluster HATs on the same network you just need to reconfigure the Controller by changing it's hostname (to controller1/controller2/etc.) and to prevent MAC address collisions the Pi Zeros need to be switched to different MAC addresses (and hostname changed to p11-p14/p21-p24/etc.). The internal 172.19.180.0/24 IP addresses will not conflict as they're isolated from the external traffic.

Setting up the software

Start by writing the Controller image to a new SD card (I'd advise using 16GB or bigger) as per the standard instructions - http://clusterhat.com/setup-software

 

I login using serial console on the Controller Pi (if you're using SSH you'll need to enable it first).

Login and do the normal first boot setup - set a ner password for the pi user, enable SSH, install the packages you'll need, download the Controller image (or copy it from your local machine) and expand the root filesystem.

sudo passwd pi
sudo touch /boot/ssh
sudo apt-get -y install git aria2 libusb-1.0-0-dev kpartx nfs-kernel-server
aria2c --seed-time 0 http://dist.8086.net/clusterhat/ClusterHAT-2017-03-02-lite-1-controller.zip.torrent
sudo raspi-config # Expand FS and reboot

 

The commands/edits below are ran as the root user.

sudo -i

 

Enable the kernel module for 802.1Q (VLAN) support.

echo 8021q >> /etc/modules

 

Edit /etc/network/interfaces and change "auto lo br0" to "auto lo brint brext"

Replace the existing "iface br0 config" section with the following config.

# Internal network (untagged on Pi Zeros)
auto brint
iface brint inet static
        bridge_ports none
        address 172.19.180.254
        netmask 255.255.255.0
        bridge_stp off
        bridge_waitport 0
        bridge_fd 0
# External network (VLAN10 on Pi Zeros)
iface brext inet manual
        bridge_ports eth0
        bridge_stp off
        bridge_waitport 0
        bridge_fd 0


Edit /etc/dhcpcd.conf and change the "denyinterfaces" line to

denyinterfaces eth0 ethpi1 ethpi2 ethpi3 ethpi4 ethpi*.10 brint

 

Alter Cluster HAT udev rules to only rename the device if it's a real USB device (and not the VLAN interface).

sed -i 's#"net", #"net", SUBSYSTEMS=="usb", #' /etc/udev/rules.d/90-clusterhat.rules

 

Generate the new network configuration for ethpiX and ethpiX.10 VLAN interfaces.

I=0
while [ $I -lt 256 ];do
echo "auto ethpi$I ethpi$I.10
allow-hotplug ethpi$I
allow-hotplug ethpi$I.10
# Internal network untagged
iface ethpi$I inet manual
        pre-up brctl addif brint ethpi$I
        up ifconfig ethpi$I up
        post-up ifup ethpi$I.10
# External network (VLAN 10)
iface ethpi$I.10 inet manual
        pre-up brctl addif brext ethpi$I.10
        up ifconfig ethpi$I.10 up
        post-up ifup ethpi$I.10"
echo
let I=$I+1
done > /etc/network/interfaces.d/clusterhat

 

Reboot the Controller Pi again to activate the changes.

reboot

 

Prepare directories for USBBOOT

mkdir -p /var/lib/clusterhat/nfs/{p1,p2,p3,p4} # NFS filesystems
mkdir /var/lib/clusterhat/boot # /boot filesystem for rpiboot tool

 

Extract the Controller image (all images are really a controller image set to reconfigure on first boot so downloading p1 would not alter the steps below).

cd /var/lib/clusterhat/
unzip ~pi/ClusterHAT-2017-03-02-lite-1-controller.zip
rm ~pi/ClusterHAT-2017-03-02-lite-1-controller.zip
mkdir mnt
LOOP=`losetup -f --show ClusterHAT-2017-03-02-lite-1-controller.img`
kpartx -av $LOOP
mount `echo $LOOP|sed s#dev#dev/mapper#`p2 mnt
mount `echo $LOOP|sed s#dev#dev/mapper#`p1 mnt/boot
tar -cC mnt . | tar -xvC nfs/p1/
umount mnt/boot
umount mnt
kpartx -dv $LOOP
losetup -d $LOOP

 

Setup files within NFS root for Pi Zeros

tar -cC nfs/p1/ . | tar -xvC nfs/p2/
tar -cC nfs/p1/ . | tar -xvC nfs/p3/
tar -cC nfs/p1/ . | tar -xvC nfs/p4/
cp -r nfs/p1/boot .
ln -s ../nfs/p4/boot/ boot/1-1.2.1
ln -s ../nfs/p3/boot/ boot/1-1.2.2
ln -s ../nfs/p2/boot/ boot/1-1.2.3
ln -s ../nfs/p1/boot/ boot/1-1.2.4
sed -i "s#^denyinterfaces.*#denyinterfaces eth0 ethpi1 ethpi2 ethpi3 ethpi4 usb0.10 usb0#" nfs/p*/etc/dhcpcd.conf
sed -i "s#^auto usb0#auto usb0.10#" nfs/p*/etc/network/interfaces
sed -i "s#^allow-hotplug usb0#allow-hotplug usb0.10#" nfs/p*/etc/network/interfaces
sed -i "s#^iface usb0 inet dhcp#iface usb0.10 inet dhcp#" nfs/p*/etc/network/interfaces
for I in 1 2 3 4 ; do
 sed -i "s/.*mmcblk0.*//" nfs/p$I/etc/fstab
 cp "/usr/share/clusterhat/cmdline.p$I" nfs/p$I/boot/cmdline.txt
 cp -f "/usr/share/clusterhat/interfaces.p" nfs/p$I/etc/network/interfaces
 cp -f "/usr/share/clusterhat/issue.p" nfs/p$I/etc/issue
 sed -i "s#^127.0.1.1.*#127.0.1.1\tp$I#g" nfs/p$I/etc/hosts
 sed -i "s/^#dtoverlay=dwc2$/dtoverlay=dwc2/" nfs/p$I/boot/config.txt
 echo "p$I" > nfs/p$I/etc/hostname
 sed -i "s#root=/dev/mmcblk0p2 rootfstype=ext4#root=/dev/nfs nfsroot=172.19.180.254:/var/lib/clusterhat/nfs/p$I rw ip=172.19.180.$I:172.19.180.254::255.255.255.0:p$I:usb0:static#" nfs/p$I/boot/cmdline.txt
 sed -i "s#MODULES=most#MODULES=netboot#" nfs/p$I/etc/initramfs-tools/initramfs.conf
 echo "BOOT=nfs" >> nfs/p$I/etc/initramfs-tools/initramfs.conf
 echo -e "dwc2\ng_cdc\nuio_pdrv_genirq\nuio\nusb_f_acm\nu_serial\nusb_f_ecm\nu_ether\nlibcomposite\nudc_core\nipv6\n" >> nfs/p$I/etc/initramfs-tools/modules
 chroot nfs/p$I/ mkinitramfs -o /boot/initramfs.img 4.4.50+
 echo "initramfs initramfs.img" >> nfs/p$I/boot/config.txt
done

 

Setup NFS server

for I in 1 2 3 4 ; do
 echo "/var/lib/clusterhat/nfs/p$I 172.19.180.$I(rw,sync,no_subtree_check,no_root_squash)" >> /etc/exports
done
systemctl enable rpcbind
systemctl restart rpcbind
systemctl enable nfs-kernel-server
systemctl restart nfs-kernel-server

 

Build the modified rpiboot tool

cd /var/lib/clusterhat
git clone https://github.com/burtyb/usbboot
cd usbboot
make

 

Enable SSH and set passwords for the Pi Zeros (this will ask for the password for each Pi Zero individually).

for I in 1 2 3 4; do
 touch nfs/p$I/boot/ssh
 echo "Updating password for Pi Zero p$I"
 chroot nfs/p$I/ passwd pi
done

 

Start rpiboot in a screen session (this could probably be setup as a service).

screen -S rpiboot -d -m /var/lib/clusterhat/usbboot/rpiboot -d /var/lib/clusterhat/boot/ -o -l -v

 

You should now be able to use the normal command to power up the Pi Zeros (with no SD cards) using.

clusterhat on

They will take quite a while to boot but you should be able to see some progress from "brctl show" and "ifconfig" when the interfaces are added and the internal 172.19.180.x IPs should start to ping after the initramfs is running.

 

Known Issues

The main issue with this setup is that after a couple of reboots the Pi Zeros stop showing up correctly on the Controller as USB devices. The work around for this is to disconnect/reconnect the USB cable to the ClusterHAT.

 

After rebooting the Controller Pi the NFS related services need to be restarted.

systemctl restart rpcbind
systemctl restart nfs-kernel-server

Also the rpiboot running in screen also needs to be restarted.

screen -S rpiboot -d -m /var/lib/clusterhat/usbboot/rpiboot -d /var/lib/clusterhat/boot/ -o -l -v

 I'm also sure the network config is wrong for the Pi Zero as it sits there waiting for longer than it probably should.

Open to suggestions/fixes/ideas over on the Google Group.

Author: Chris Burton
Last update: 2019-07-19


[LEGACY] Guide to using the RPIBOOT [test] image on the Cluster HAT/Zero Dongle / Zero Stem or just a USB cable.

See the current documentation on booting without SD cards, this legacy information is kept here in case it helps anyone.

 

Cluster HAT usbboot/rpiboot test image (Updated 2019/05/15).

This is a guide to using my modified Raspbian image with rpiboot (usbboot) for a Cluster HAT with up to 4 Pi Zeros (1.1/1.2/W) which can be  modified to work with a ZeroDonfle/ZeroStem or standard USB cable.

This test image currently supports 4 Pi Zeros.

 

 

What is usbboot/rpiboot?

The "rpiboot" tool allows you to boot Raspberry Pi Zero (and Pi Zero W, Model A+, CM, CM3, CM3+ and "recent" 3A+) without an SD card, "usbboot" is the repository which holds the "rpiboot" tool. The sources are available at https://github.com/raspberrypi/usbboot .

The standard rpiboot supports booting from a single "boot" directory. My version of rpiboot has been modified to support overlay boot directories based on the USB "path" the device is connected via. This allows the use of a custom configuration file for each slot on the Cluster HAT (or ZeroDongle/ZeroStem/USB Cable/etc). This is used to give each Pi a custom cmdline.txt which sets the MAC address on both sides of the USB Gadget Ethernet device, mount the right NFS directory, etc.

Download

ClusterHAT-2019-04-08-lite-1-usbboot.zip (Stretch for Cluster HAT on RPi (boots Zero/CM1/CM3/CM3+ and "recent" 3A+ as USB devices).

http://dist.8086.net/clusterhat/ClusterHAT-2019-04-08-lite-1-usbboot.zip

ClusterHAT-2018-10-13-lite-1-usbboot.zip (Stretch for Cluster HAT v1.x and v2.x on RPi upto 3B+).

This image was released on 2018/10/13 to add support for Raspberry Pi 3B+.

 http://dist.8086.net/clusterhat/ClusterHAT-2018-10-09-lite-1-usbboot.zip

ClusterHAT-2017-11-29-lite-2-usbboot.zip (Stretch for Cluster HAT v1.x and v2.x).

This image was released on 2018/03/03 to add support for newer v2.x Cluster HAT which uses a different "on" signal for the USB HUB.

 http://dist.8086.net/clusterhat/ClusterHAT-2017-11-29-lite-2-usbboot.zip

ClusterHAT-2017-09-07-lite-1-usbboot (Stretch for Cluster HAT v1.x and v2.0).

A new Stretch based image released on 2017/11/27 which no longer needs to be upgraded to support the Cluster HAT v2.0 can be downloaded from:

 http://dist.8086.net/clusterhat/ClusterHAT-2017-09-07-lite-1-usbboot.zip

I advise using a 16GB SD card as the image requires 7.5GB, the image will be automatically expanded to fill the SD card on first boot.

ClusterHAT-2017-07-05-lite-2-usbboot (Jessie for Cluster HAT v1.x)

The older ClusterHAT-2017-07-05-lite-2-usbboot.zip Jessie based image can still be downloaded from:

 http://dist.8086.net/clusterhat/ClusterHAT-2017-07-05-lite-2-usbboot.zip

I advise using a 16GB SD card as the image requires 5GB and MUST be expanded with "sudo raspi-config" on the controller before trying to boot any Pi Zero.

Issue with ClusterHAT-2017-07-05-lite-1-usbboot.zip - If you've downloaded the earlier version of this then you may find rpiboot gets stuck after booting the first Pi Zero. To fix this you need to update rpiboot by running the following command on the Controller Pi and then reboot.

sudo sh -c "cd /var/lib/clusterhat/usbboot/;git pull;make"

When using either OLD 2017-07-05 image with a Cluster HAT v2.0 you will also need to upgrade the tools to support the newer HAT - see the forum post for upgrade instructions.

Installation

The image should be written to an SD card for your Controller Raspberry Pi by following the standard instructions.

Stretch based images have SSH disabled by default (on Controller/Pi Zeros) and the filesystem will automatically expand.

Older Jessie based image have SSH enabled by default on both the Controller and Pi Zeros, the Controller filesystem must be expanded on first boot using "sudo raspi-config".

The "pi" users password is set to "clusterhat" (without the quotes) you MUST change this as soon as you login AND then expand the Controller filesystem with "sudo raspi-config".

Also please remember to remove SD cards from the Pi Zeros :).

Network Overview

The Pi Zeros have an internal network bridged to "brint" on the Controller, they also have an external network bridged to "brext" on the Controller. The Pi Zeros use VLANs to separate these 2 networks. Untagged traffic is used for internal and VLAN 10 is used for external traffic. It is done this way arround to utilise mounting the NFSROOT filesystem from the kernel command line where configuring VLAN is not possible.

See our original post for more details on this setup.

Internal Network

172.19.180.254 - Is assigned to the Controllers Internal Interface.

172.19.180.1-4 - Are assigned to the Pi Zeros.

These can be used if the Pi Zero doesn't pick up an external IP address correctly or if you want to boot it standalone.

External Network

The Controller/Pi Zeros will request the external IP address via DHCP on the Controllers "eth0" device.

The hostnames are "controller.local", "p1.local", "p2.local", "p3.local", "p4.local".

Usage [Cluster HAT]

The image as supplied expects the Cluster HAT to be plugged into the top USB port closest to the Ethernet port (for 3B) and bottom USB port furthest from the Ethernet port (for 3B+) on the Controller Pi.

If using a Cluster HAT P1-P4 can be powered on using the standard "clusterhat on" command. 

You should be able to watch the Pi Zero request files and boot from by looking at the screen session for "rpiboot".

sudo screen -rx rpiboot
# To detach from screen press CTRL+a then CTRL+d

Once connected it will show files being requested as it steps through the boot process and eventually after a few seconds the LED should illuminate on the Pi Zero and it will boot up. It may take more than a minute before the Pi Zero requests the external IP address and starts SSH but you should be able to ping 172.19.180.1/2/3/4 early in the boot process as this is used to mount the root filesystem over NFS.

Pi Zeros can be rebooted as normal and will be booted up again automatically.

Usage [ZeroDongle / ZeroStem / USB Cable]

This section is a bit long/more complicated than it needs to be right now, will be cleaned up - if you need help post on the forum (If you can post the output of "lsusb -t" before and after you plug in the Pi Zero it should be easy to spot even when using standard Raspbian).

The root file system for the Pi Zeros is stored in /var/lib/clusterhat/nfs/pX where X is 1-4. The base "/boot" filesystem is stored in /var/lib/clusterhat/boot/ and within this directory are symlinks to the specific root filesystems for each USB path.

ls -l /var/lib/clusterhat/boot/1-*
lrwxrwxrwx 1 root root 15 Aug  2 09:11 /var/lib/clusterhat/boot/1-1.2.1 -> ../nfs/p4/boot/
lrwxrwxrwx 1 root root 15 Aug  2 09:11 /var/lib/clusterhat/boot/1-1.2.2 -> ../nfs/p3/boot/
lrwxrwxrwx 1 root root 15 Aug  2 09:11 /var/lib/clusterhat/boot/1-1.2.3 -> ../nfs/p2/boot/
lrwxrwxrwx 1 root root 15 Aug  2 09:11 /var/lib/clusterhat/boot/1-1.2.4 -> ../nfs/p1/boot/

These relate to the USB path names so in the example above this relates to.

1-1.2 - Is the top left USB port on the Raspberry Pi 3 which is connected to the USB hub on the Cluster HAT.
1-1.2.4 - Is the first USB port on the Cluster HAT which is used for P1.
1-1.2.3 - Is the second USB port on the Cluster HAT which is used for P2.
1-1.2.2 - Is the third USB port on the Cluster HAT which is used for P3.
1-1.2.1 - Is the fourth USB port on the Cluster HAT which is used for P4.

To use a different USB path topology you'll need to alter these symlinks, first find out the USB path name where you're going to be plugging in the Pi Zero.

So if you're plugging in a Pi Zero with a USB cable or Zero Dongle / Zero Stem directly into...

Top Left USB port on a Pi 3 the USB pathname would be "1-1.2"
Top Right USB port on a Pi 3 the USB pathname would be "1-1.4"
Bottom Left USB port on a Pi 3 the USB pathname would be "1-1.3"
Bottom Right USB port on a Pi 3 the USB pathname would be "1-1.5"

You can also see the path using the following command.

 

$ lsusb -t
/:  Bus 01.Port 1: Dev 1, class="root_hub", Driver=dwc_otg/1p, 480M
    |__ Port 1: Dev 2, If 0, class="Hub", Driver=hub/5p, 480M
        |__ Port 1: Dev 3, If 0, class="Vendor" Specific Class, Driver=smsc95xx, 480M
        |__ Port 2: Dev 4, If 0, class="Hub", Driver=hub/4p, 480M
            |__ Port 3: Dev 33, If 2, class="Communications", Driver=cdc_acm, 480M
            |__ Port 3: Dev 33, If 0, class="Communications", Driver=cdc_ether, 480M
            |__ Port 3: Dev 33, If 3, class="CDC" Data, Driver=cdc_acm, 480M
            |__ Port 3: Dev 33, If 1, class="CDC" Data, Driver=cdc_ether, 480M
            |__ Port 1: Dev 47, If 3, class="CDC" Data, Driver=cdc_acm, 480M
            |__ Port 1: Dev 47, If 1, class="CDC" Data, Driver=cdc_ether, 480M
            |__ Port 1: Dev 47, If 2, class="Communications", Driver=cdc_acm, 480M
            |__ Port 1: Dev 47, If 0, class="Communications", Driver=cdc_ether, 480M
            |__ Port 4: Dev 49, If 1, class="CDC" Data, Driver=cdc_ether, 480M
            |__ Port 4: Dev 49, If 2, class="Communications", Driver=cdc_acm, 480M
            |__ Port 4: Dev 49, If 0, class="Communications", Driver=cdc_ether, 480M
            |__ Port 4: Dev 49, If 3, class="CDC" Data, Driver=cdc_acm, 480M
            |__ Port 2: Dev 44, If 2, class="Communications", Driver=cdc_acm, 480M
            |__ Port 2: Dev 44, If 0, class="Communications", Driver=cdc_ether, 480M
            |__ Port 2: Dev 44, If 3, class="CDC" Data, Driver=cdc_acm, 480M
            |__ Port 2: Dev 44, If 1, class="CDC" Data, Driver=cdc_ether, 480M

 The 1-1.2.4 relates to "Bus 01" - "Port 1" . "Port 2" . "Port 4" (which is Dev 49 in the example above) so by plugging something into the port and running "lsusb -t" before/after you should be able to work out the path name you need.

So for example to change /var/lib/clusterhat/nfs/p1 to be the filesystem for a Pi zero plugged into the bottom right USB port on a Pi 3 you would need to update the symlink to "1-1.5".

sudo rm -f /var/lib/clusterhat/boot/1-1.2.4
sudo ln -s /var/lib/clusterhat/nfs/p1/boot/ /var/lib/clusterhat/boot/1-1.5

If you have any problem working out the path name you should be using please ask on the forum.

Hacks

The rpiboot process runs in a screen session (under the root user) rather than a systemd service. I do this to make it easier to monitor what's going on when needed without having multiple lines per second end up in the log file. Once things stabalise I expect this to move to a systemd service. This is started in the /etc/rc.local file.

There are still issues with the rpcbind/nfs services not starting on boot so again these are restarted in the /etc/rc.local file which seemed to work OK in my testing.

raspi-config doesn't work as it tries to detect the "/boot" partition is a mountpoint which is isn't as it comes with the root filesystem you can appease it by mounting the "/boot" directory again on the Pi Zero.

sudo mount 172.19.180.254:/var/lib/clusterhat/nfs/p1/boot /boot

Replacing the "p1" as appropriate, you should then be able to run raspi-config to enable the camera/i2c/etc.

Problems/Questions?

If you have any problems, questions or suggestions about the images please use the forum to discuss it.

How?

If you want to make your own image or see how I created this image the script is available here but be aware the script has bugs, it expects to be ran with a ClusterHAT image as the source file on a Raspberry Pi with plenty of storage.

Author: Chris Burton
Last update: 2019-07-19


How do I copy SSH keys to P1-P4 when using usbboot/rpiboot?

Since the filesystems for P1-P4 are stored on the Controller it's even easier to setup SSH keys to log into the Pi Zeros without using a password from the Controller.

ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa
mkdir -p /var/lib/clusterctrl/nfs/p{1,2,3,4}/home/pi/.ssh/
cat ~/.ssh/id_rsa.pub >> /var/lib/clusterctrl/nfs/p1/home/pi/.ssh/authorized_keys
cat ~/.ssh/id_rsa.pub >> /var/lib/clusterctrl/nfs/p2/home/pi/.ssh/authorized_keys
cat ~/.ssh/id_rsa.pub >> /var/lib/clusterctrl/nfs/p3/home/pi/.ssh/authorized_keys
cat ~/.ssh/id_rsa.pub >> /var/lib/clusterctrl/nfs/p4/home/pi/.ssh/authorized_keys

You should now be able to log into the Pi Zeros.

ssh pi@p1.local

To run a command on all Pi Zeros sequentially you can reuse a portion of the above command.

for I in 1 2 3 4; do echo -n "p$I:";ssh pi@p$I.local uptime;done

You should see something similar to the output below which shows how long each Pi Zero has been booted, number of users, etc.

p1: 21:43:32 up 24 min,  3 users,  load average: 0.01, 0.02, 0.00
p2: 21:43:33 up 6 min,  2 users,  load average: 0.07, 0.08, 0.05
p3: 21:43:33 up 5 min,  2 users,  load average: 0.00, 0.07, 0.04
p4: 21:43:34 up 4 min,  2 users,  load average: 0.02, 0.11, 0.06

Author: Chris Burton
Last update: 2020-07-02


How do I enable autologin on TTY running on the Controllers serial port [Raspbian Stretch]?

To enable auto logins on the TTY running on any serial port on the Controller log in and run the following command and reboot.

sudo sed -i "s#agetty --keep-baud#agetty --autologin pi --keep-baud#" /lib/systemd/system/serial-getty@.service

And to disable it run the following command and reboot.

sudo sed -i "s#agetty --autologin pi --keep-baud#agetty --keep-baud#" /lib/systemd/system/serial-getty@.service

To login automatically using a different username replace the "pi" username in the above commands.

When enabling this all users with access to the Controller Pi serial port will be able to log in without any authentication.

Author: Chris Burton
Last update: 2019-09-09


How do I setup NAT on the WiFi interface on the rpiboot test images?

WARNING: These steps relate to the OLD test rpiboot (only) images and NOT the current combined images, if you want NAT on the current image just switch the type to cnat (see Intermediate tab on https://clusterctrl.com/setup-software to switch from CBRIDGE).

 

Setup the wireless connection as normal by following the documentation.

Install the package to allow saving the iptables rules, say no to saving existing ipv4/ipv6 rules as you won't have any.

sudo apt-get -y install iptables-persistent

Add a default gateway to the IP configuration for the Pi Zeros

sudo sed -i "s/172.19.180.254::255.255.255.0/172.19.180.254:172.19.180.254:255.255.255.0/" /var/lib/clusterhat/nfs/p*/boot/cmdline.txt

Enable IPv4 forwarding

sudo sh -c "echo 'net.ipv4.ip_forward=1' >> /etc/sysctl.conf"

sudo sysctl -p /etc/sysctl.conf

Setup and save the NAT rules for traffic from the Pi Zero internal network range.

sudo iptables -t nat -A POSTROUTING -s 172.19.180.0/24 ! -o brint -j MASQUERADE

sudo iptables -A FORWARD -i brint ! -o brint -j ACCEPT

sudo iptables -A FORWARD -o brint -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

sudo sh -c "iptables-save > /etc/iptables/rules.v4"

You should now be able to power up the Pi Zeros and access the internet through the wifi connection on the controller.

Author: Chris Burton
Last update: 2020-09-17


How do I enable mDNS/zeroconf/Bonjour on Armbian?

To allow <hostname>.local domains to be resolved on Armbian install the following.

sudo apt-get install avahi-daemon libnss-mdns libnss-mymachines

You should then be able to resolve domains using mDNS.

Author: Chris Burton
Last update: 2019-05-22


How do I USBBOOT Pi Zero/Pi Zero 2 without SD cards (Cluster HAT/Cluster CTRL)?

With the Cluster CTRL images (linked from the ClusterHAT website) it's now possible to mix/match booting nodes (Pi Zeros/Compute Modules/A+) with or without SD cards.

Preparation

This guide assumes you're running a Raspberry Pi with our modified Raspbian Bullseye/Buster/Stretch ClusterCTRL images (see the Cluster HAT site for details).

For the ClusterHAT v1.x edit /etc/default/clusterctrl on the controller and uncomment the appropriate CLUSTERHATV1 line based on the USB port you have the Cluster HAT plugged into, then run "sudo clusterctrl init". 

The Cluster HAT v2.x/ClusterCTRL devices don't need this step as the USB port is detected as the controller boots.

If the port is changed after boot (for v1.x you'll need to set the correct port again first) run "sudo clusterctrl init" to setup the boot directory links in /var/lib/clusterctrl/boot/ .

Setup Pi Zero file systems

Differing from the old test images the filesystems for the Pi Zeros are no longer pre-installed. This allows you to mix/match different versions and releases (Bullseye/Buster/Stretch) on the same controller.

Official ClusterCTRL usbboot archives can be found at https://dist.8086.net/clusterctrl/usbboot/

I would advise using the latest dated Bullseye release and then choose if you want Lite, Full or Standard image. There are also Buster/Stretch usbboot root filesystems available if you need to use an older release.

Once you've selected the image(s) you want to use copy the URL and download them to the controller using wget.

wget -P ~pi/ URL_OF_USBBOOT_ARCHIVE

For example to download the 2022-04-04 Lite Bullseye image you would run

wget -P ~pi/ https://dist.8086.net/clusterctrl/usbboot/bullseye/2022-04-04-2-buster-ClusterCTRL-armhf-lite-usbboot.tar.xz

For each Pi Zero (in position pX) extract the filesystem (you can mix/match different versions) and then configure it.

If you have files in the nodes root filesystem these must be removed first (replacing X with the pX number). Ensure you have a backup of any required files first!

sudo rm -rf /var/lib/clusterctrl/nfs/pX/{*,.*}

The error "unable to remove  directories . and .." error is expected and can be ignored.

For example for P1 you would extract and configure it using.

sudo tar -axf ~pi/2022-04-04-2-buster-ClusterCTRL-armhf-lite-usbboot.tar.xz -C /var/lib/clusterctrl/nfs/p1/
sudo usbboot-init 1

Repeat the above two commands for any other Pi Zeros 2/3/4/etc. where you want to use USBBOOT. The archives aren't specific to P1/P2/etc. (unlike the SD images) but they need to be converted by running "usbboot-init X" replacing X with the Px number. You can mix booting with SD/USBBOOT with Bullseye/Buster/Stretch releases and 32/64-bit where they support running on the Pi Zero/2 hardware.

Whilst older stretch/buster archives have default login details (user: pi pass: clusterctrl) newer images do not have a default username pr passwords. For newer bullseye usbboot archies please use the following steps (replacing pX with p1/p2/etc. and myusername/mypassword with your desired username/password) to setup a username and password before the first boot.

set +o history # Turn off bash history temporarily
# Replace myusername and mypassword with your details and pX with the node (p1/p2/etc.).
echo "myusername:"$(echo 'mypassword' | openssl passwd -6 -stdin) | sudo tee /var/lib/clusterctrl/nfs/pX/boot/userconf.txt
set -o history # Turn history back on again

Before powering on the Pi Zero you can enable SSH by creating the "ssh" file in the "/boot" directory.

sudo touch /var/lib/clusterctrl/nfs/p1/boot/ssh

Power the Pi Zero on as normal.

clusterctrl on p1

You can watch the boot process by tailing /var/log/daemon.log (shows rpiboot process) /var/log/kern.log (shows USB devices etc)

tail -f /var/log/daemon.log /var/log/kern.log

 

Author: Chris Burton
Last update: 2023-06-26


How do I change a Cluster CTRL image type (Controller/NAT/P1/P2/P3/P4/etc.)?

If you're using an older style Cluster HAT image (if you have /etc/default/clusterctrl then you're using the new style), follow the ClusterHAT instructions.

On the Cluster HAT images there are three types of image.
cbridge - Bridges eth0 on the controller to usb0 on the nodes (Pi Zeros).
cnat - Pi Zero have network access via either WiFi or Ethernet on the controller via NAT (Network Address Translation).
pX - Node image for each Pi Zero (replace X with the Pi Zero number, p1, p2, p3, p4, etc.).

All ClusterCTRL images start out life as a "cbridge" image, they're then reconfigured on first boot to be either of the types above.

To change the type edit the cmdline.txt file in the boot partition, this can be done either on a booted system "/boot/cmdline.txt" or from another machine editing the file on the USB/SD card.

Append "init=/sbin/reconfig-clusterctrl X" to the end of the line replacing X with the type above (this must be a single line).

During the next boot the Pi will reconfigure and change the hostname to cbridge/cnat/p1/p2/p3/p4/etc.

Author: Chris Burton
Last update: 2019-07-19


How do I change a Cluster HAT image type (Controller/NAT/P1/P2/P3/P4/etc.)?

If you're using an newer style Cluster CTRL image (the file /etc/default/clusterctrl exists) please follow Cluster CTRL instructions.

On the Cluster HAT images there are three types of image.
controller - Bridges eth0 on the controller to usb0 on the nodes (Pi Zeros).
cnat - Pi Zero have network access via either WiFi or Ethernet on the controller via NAT (Network Address Translation).
pX - Node image for each Pi Zero (replace X with the Pi Zero number, p1, p2, p3, p4, etc.).

All ClusterHAT images start out life as a "controller" image, they're then reconfigured on first boot to be either of the types above.

To change the type edit the cmdline.txt file in the boot partition, this can be done either on a booted system "/boot/cmdline.txt" or from another machine editing the file on the USB/SD card.

Append "init=/sbin/reconfig-clusterhat X" to the end of the line replacing X with the type above (this must be a single line).

During the next boot the Pi will reconfigure and change the hostname to controller/cnat/p1/p2/p3/p4/etc.

Author: Chris Burton
Last update: 2019-07-19


How do I resize a disk image (Raspbian SD/USB image)?

To resize the last partition in a disk image, for example a Raspberry Pi SD card image first increase the size of the file on disk using truncate, here we're going to increase it by 500MB.

truncate 2019-06-20-raspbian-buster-full.img --size=+500M

Once the file size has been increased the size of the second partition need to be updated to use all of the remaining space.

parted --script ./2019-06-20-raspbian-buster-full.img resizepart 2 100%

And then the filesystem will need to be extended to the size of the new partition, first enable access to the partitions in the disk image.

losetup -fP --show 2019-06-20-raspbian-buster-full.img

This will return the loop device name, in this example we're going to assume it's "/dev/loop1". 

Then check the filesystem.

e2fsck -f /dev/loop1p2

Now we can do the actual filesystem resize.

resize2fs /dev/loop1p2

If you see no errors then we're done and we can remove the loop interface.

losetup -d /dev/loop1

The size of the disk image has now been extended and the second partition has been increased to use all of the space.

Author: Chris Burton
Last update: 2019-07-24


When using CNAT the br0 (controller) or usb0 (node) doesn't get an IP via DHCP or set the fallback IP.

From testing I have seen this happen when dhcpcd is unable to obtain an IP address for the interface before systemd classes it as a timeout and kills the service.

This can be fixed by increasing the time systemd lets the service run before terminating it.

sudo systemctl edit --full dhcpcd5.service

After the last line in the [Service] section append "TimeoutStartSec=900" and save the file and then reload the changes with.

sudo systemctrl daemon-reload

After rebooting you should now either get an IP via DHCP or use the fallback IP.

Author: Chris Burton
Last update: 2019-08-03


Why do I get the fallback IP instead of an IP via DHCP?

The time dhcpcd waits before using the fallback IP can be set in the configuration file.

Edit /etc/dhcpdcd.conf

Add a line containing "timeout 60" (increasing from the default of 30 seconds) to the start of the file so it looks like this.

# A sample configuration for dhcpcd.
# See dhcpcd.conf(5) for details.

timeout 60

# Allow users of this group to interact with dhcpcd via the control socket
#controlgroup wheel

And then reboot.

If after a reboot you don't get any IP address on the Pi Zero(node) then you may need to increase the systemd timeout as it may be killing the service.

Author: Chris Burton
Last update: 2019-08-03


ClusterCTRL: How to write an image to the eMMC?

To write an image to the eMMC on a Compute Module you need to boot it as a USB Device using the MSD (Mass Storage Device) files provided with the rpiboot tool.

In the following steps you will need to replace X with the Pi node number (X=1 for p1, X=5 for p5, etc).

Background

When using ClusterCTRL with Compute Modules "clusterctrl status" will show the status for both Power "pX:S" (X=Node number, S=Status 0=on/1=off) and USBBOOT "uX:S" (X=Node number, S=Status on/off).

uX:0 Try to boot from the onboard eMMC and fallback to booting as a USB device.
uX:1 Boot as a USB device.

You will also see a "ctrl_bus:" line, this shows the details for all ClusterCTRL device found (space separate if you have multiple ClusterCTRL devices).

For each ClusterCTRL device you will see 3 numbers, for example.

ctrl_bus:20:3:3

The first number above "20" shows the ORDER, this is used if you have multiple ClusterCTRL devices on the same controller (lower numbers have a higher priority). A ClusterHAT always has highest priority so if present will be used for p1-p4. It will then assign the next pX.. numbers to the ClusterCTRL device with the lowest ORDER, then the next lowest etc (you may need this ORDER value to save the state below).

The second number "3" is the I2C bus (/dev/i2c-3 for example) used to communicate with the ClusterCTRL device.

The last number "3" is the maximum number of nodes this device supports.

Enable USBBOOT

First power off the node with "clusterctrl off pX" then check the USBBOOT setting with "clusterctrl status".

If you see "uX:1" you don't need to make any changes to write the image.
If you see "uX:0" run "clusterctrl usbboot on pX" to enable USBBOOT.

If you've moved the USB Cable since powering on the controller you will need to re-run "sudo clusterctrl init" to reconfigure the symlinks for USBBOOT.

Setup Mass Storage Device boot files

Make sure the boot directory for pX is empty (this will remove all files for the pX filesystem, ensure you have a backup of any data first).

sudo rm -rf /var/lib/clusterctrl/nfs/pX/{*,.*}

The error "unable to remove  directories . and .." error is expected and can be ignored.

Then create the boot directory and copy over the files required to boot as a MSD.

sudo mkdir /var/lib/clusterctrl/nfs/pX/boot/
sudo cp -r /usr/share/rpiboot/msd/ /var/lib/clusterctrl/nfs/pX/boot/

Power on the Compute Module

clusterctrl on pX

You can watch the Compute Module booting as a USB device by tailing the log (CTRL-c to exit).

tail -f /var/log/kern.log /var/log/daemon.log

In a few seconds you should see a new MSD /dev/sdY - please remember if you have other storage devices on your Pi this may not always be the same device (/dev/sda for example).

You can now use "dd" as normal to write an image to this device, in the example below I'm going to call the device /dev/sdY you will need to replace sdY with the device shown in the logs above. Remember getting this wrong may overwrite another storage device or just create a file on the root filesystem!

sudo dd if=ClusterCTRL-2019-07-10-lite-1-p4.img of=/dev/sdY bs=4M status=progress

Set username and password / reconfigure node number / enable SSH

Recent Raspberry Pi OS require a username and password to be set, they no longer have the default "pi" user. You may also need to change the pX number of the node (for example to change a p4 image to p5). Or enable SSH which is also disabled by default on current Raspberry Pi OS.

Reread the partition table (replace sdY with the real device).

sudo partprobe /dev/sdY

Mount the boot filesystem (replace sdY with the real device).

sudo mount /dev/sdY1 /mnt/

To set a username and password (required)

set +o history # Turn off bash history temporarily
# Replace myusername and mypassword with your details.
echo "myusername:"$(echo 'mypassword' | openssl passwd -6 -stdin) | sudo tee /mnt/userconf.txt
set -o history # Turn history back on again

To change the pX node number edit cmdline.txt changing the end of the "init=" part to the desired pX at the end of the line.

sudo nano /mnt/cmdline.txt

Ensure cmdline.txt is a single line.

To enable SSH

sudo touch /mnt/ssh

Unmount the boot filesystem.

sudo umount /mnt

Clean up

Force all blocks to be written.

sudo sync

Power off the node and disable USBBOOT

clusterctrl off pX
clusterctrl usbboot off pX

Save power on state (optional)

If you want the node to always boot from eMMC you will need to save the state (replacing ORDER with the order number shown in the "clusterctrl status" output above).

clusterctrl save ORDER

This command saves both power and USBBOOT states for use on next power up

You can now power on the node and it will boot from the eMMC.

clusterctrl on pX

Within a few seconds you should see it boot up (the first boot will be slower as it reconfigures and resizes the filesystem).

tail -f /var/log/kern.log /var/log/daemon.log

You can now accesses the node as normal "ping pX.local", "ssh pi@pX.local", if you haven't enabled SSH you can access it using the serial console "screen /dev/ttypiX" (CTRL-x then 'k' to exit).

Author: Chris Burton
Last update: 2023-12-02


ClusterCTRL: How do I boot a Compute Module from eMMC?

To boot a Compute Module from the eMMC USBBOOT must be disabled.

If your Compute Module doesn't already have a bootable system see our other instructions.

Background

When using ClusterCTRL with Compute Modules "clusterctrl status" will show the status for both Power "pX:S" (X=Node number, S=Status 0=on/1=off) and USBBOOT "uX:S" (X=Node number, S=Status on/off).

uX:0 Try to boot from the onboard eMMC and fallback to booting as a USB device.
uX:1 Boot as a USB device.

You will also see a "ctrl_bus:" line, this shows the details for all ClusterCTRL device found (space separate if you have multiple ClusterCTRL devices).

For each ClusterCTRL device you will see 3 numbers, for example.

ctrl_bus:20:3:3

The first number above "20" shows the ORDER, this is used if you have multiple ClusterCTRL devices on the same controller (lower numbers have a higher priority). A ClusterHAT always has highest priority so if present will be used for p1-p4. It will then assign the next pX.. numbers to the ClusterCTRL device with the lowest ORDER, then the next lowest etc (you may need this ORDER value to save the state below).

The second number "3" is the I2C bus (/dev/i2c-3 for example) used to communicate with the ClusterCTRL device.

The last number "3" is the maximum number of nodes this device supports.

Disable USBBOOT

Power of the node and disable USBBOOT.

clusterctrl off pX
clusterctrl usbboot off pX

The next time the node is powered on it will boot from onboard eMMC.

Save power on state (optional)

If you want the node to always boot from eMMC you will need to save the state (replacing ORDER with the order number shown in the "clusterctrl status" output above).

clusterctrl save ORDER

This command saves both power and USBBOOT states for use on next power up

Power on

You can now power on the node and it will boot from the eMMC.

clusterctrl on pX

Within a few seconds you should see it boot up in the logs, the first boot will be slower as it reconfigures and resizes the filesystem (CTRL-c to exit).

tail -f /var/log/kern.log /var/log/daemon.log

You can now accesses the node as normal "ping pX.local", "ssh pi@pX.local", if you haven't enabled SSH you can access it using the serial console "screen /dev/ttypiX" (CTRL-x then 'k' to exit).

Author: Chris Burton
Last update: 2019-08-09


ClusterCTRL: How do I USBBOOT a Compute Module?

To boot a Compute Module as a USB device USBBOOT must be enabled.

Background

When using ClusterCTRL with Compute Modules "clusterctrl status" will show the status for both Power "pX:S" (X=Node number, S=Status 0=on/1=off) and USBBOOT "uX:S" (X=Node number, S=Status on/off).

uX:0 Try to boot from the onboard eMMC and fallback to booting as a USB device.
uX:1 Boot as a USB device.

You will also see a "ctrl_bus:" line, this shows the details for all ClusterCTRL device found (space separate if you have multiple ClusterCTRL devices).

For each ClusterCTRL device you will see 3 numbers, for example.

ctrl_bus:20:3:3

The first number above "20" shows the ORDER, this is used if you have multiple ClusterCTRL devices on the same controller (lower numbers have a higher priority). A ClusterHAT always has highest priority so if present will be used for p1-p4. It will then assign the next pX.. numbers to the ClusterCTRL device with the lowest ORDER, then the next lowest etc (you may need this ORDER value to save the state below).

The second number "3" is the I2C bus (/dev/i2c-3 for example) used to communicate with the ClusterCTRL device.

The last number "3" is the maximum number of nodes this device supports.

Enable USBBOOT

Power of the node and disable USBBOOT.

clusterctrl off pX
clusterctrl usbboot on pX

The next time the node is powered on it will boot as a USB device.

Save power on state (optional)

If you want the node to always boot using USBBOOT you will need to save the state (replacing ORDER with the order number shown in the "clusterctrl status" output above).

clusterctrl save ORDER

This command saves both power and USBBOOT states for use on next power up

You can now follow our standard instructions on using USBBOOT.

Author: Chris Burton
Last update: 2019-08-06


ClusterCTRL: How do I USBBOOT a Compute Module and use eMMC as local storage/swap?

A Compute Module will boot as a USB device if either USBBOOT is enabled or it can't find bootcode.txt in the first FAT partition on the eMMC.

Replace X in the following steps with X of the pX device you're configuring (for X=1 for p1, X=5 for p5, etc).

First setup the filesystem and Compute Module to boot as a USB Device.

Once USBBOOT is working power on the Compute Module "clusterctrl on pX" and wait for it to boot - (until you can "ping pX.local", etc).

On the controller you will now need to disable USBBOOT.

clusterctrl usbboot off pX

To permanently allow access to the eMMC as local storage you will need to save the power on state (see the USBBOOT instructions on how to do this).

Log into the Compute Module node (via either "ssh pi@pX.local" or "screen /dev/ttypiX").

You can now access the eMMC as /dev/mmcblk0 which can be partitioned with fdisk/etc.

The existing partitions can be viewed with fdisk

sudo fdisk -l /dev/mmcblk0

To ensure the Compute Module boots as a USB Device (and not from the eMMC) you will need to ensure the first FAT partition does not contain bootcode.bin file (either by removing the file or not having a FAT partition at all).

For example to use it as a 1GB swap partition and the rest as a /data partition you could do the following.

Using fdisk remove the existing partitions (this will remove all existing data so backup first).

sudo fdisk /dev/mmcblk0

Press "p" and return to see the exising partitions.

For each existing partition press "d" and return, enter the partition number or press enter to delete the last one each time.

Once the existing partitions are removed you can add the new partitions.

Type "n" and return, use "p" for a primary partition, use parition number 1, it will start with the first available sector so you can press enter, for the size enter "+1G". To set the type to "Linux Swap" press "t" return, then type "82" (Linux Swap) and press return.

Then add the data partition, again type "n" and return, use "p" for a primary partition, use parition number 2, go with the default first and end sectors by pressing enter twice.

You can then use "p" and enter to see the partitions.

If you're happy use "w" and return to write the changes and exit or "q" to quit without changing the partition table.

Reload the partition table

sudo partprobe

Setup swap and create the filesystem

sudo mkswap /dev/mmcblk0p1

This will show you the UUID for the swap partition, keep this for later.

sudo mke2fs -j /dev/mmcblk0p2

Again this will show the UUID you will need later.

Create the mount point for the filesystem, here I'm using /data

sudo mkdir /data

Edit /etc/fstab to setup using both swap and the new filesystem

sudo nano /etc/fstab

Append new lines for both swap and data partitions (replacing with your UUID from above).

UUID=029577ad-b1d2-43ad-b1f9-348e37969049       swap    swap defaults 0 0
UUID=e5dd1581-8baf-43d7-b49b-a24cb93da970       /data   ext4 defaults 0 0

Write the file and exit nano.

Check the swap partition is setup OK in /etc/fstab

sudo swapon -a

You should now see the added swap space if you run "free".

Mount the filesystem.

sudo mount -a

You should see the mounted filesystem by running "df -h".

After rebooting the Compute Module it will still boot via USBBOOT and use the new local swap and /data filesystem.

Author: Chris Burton
Last update: 2019-08-06


What does the output of 'clusterctrl status' mean?

Running "clusterctrl status" you will get output similar to the below text.

clusterhat:1
clusterctrl:2
maxpi:10
ctrl_bus:20:4:3 21:3:3
hat_version:1.2
hat_version_major:1
hat_version_minor:2
hat_size:4
hat_uuid:622aef87-17d6-4022-a370-781e1c3fe2b9
hat_vendor:8086 Consultancy
hat_product_id:0x0004
throttled:0x0
hat_alert:0
p1:0
p2:0
p3:0
p4:0
ctrl20:FW:1.1 ADC1:5014mV T1:18.60C
p5:0
u5:1
p6:0
u6:1
p7:0
u7:1
ctrl21:FW:1.1 ADC1:4891mV T1:23.52C
p8:0
u8:1
p9:0
u9:1
p10:0
u10:1

clusterhat:
False if no Cluster HAT is detected.
1 if a Cluster HAT was detected.

clusterctrl:
False if no Cluster CTRL devices (Single/Triple/Stack/A+6) detected
X number of devices detected

maxpi:
A count of the number of nodes controllable.

ctrl_bus:
Space separated list of data (order:I2C device:maxpi) from each Cluster CTRL device.

From the example above
20:4:3 - Cluster CTRL device with order 20, is controllable via /dev/i2c-4 and controls 3 nodes.
21:3:3 - Cluster CTRL device with order 21, is controllable via /dev/i2c-3 and controls 3 nodes.

hat_version:
Full version number of the Cluster HAT detected (1.2, 2.3, etc).

hat_version_major:
Major Cluster HAT version (1 for 1.x, 2 for 2.x, etc).

hat_version_minor:
Minor part of the version number (4 for 1.4, 3 for 2.3, etc).

hat_size:
Number of Pi Zeros in the Cluster HAT this is by default 4, the maximum number supported. It can be overriden by editing /etc/defaults/clusterctrl and changing the value for CLUSTERHAT_SIZE. This will change the number of nodes controlled by the Cluster HAT for example if you only have 3 Pi Zero you can use CLUSTERHAT_SIZE=3 and the p4 name would be used for Cluster CTRL nodes.

hat_uuid:
The UUID for the Cluster HAT device, unique for every product.

hat_vendor:
Vendor name "8086 Consultancy".

hat_product_id:
8086 Consultancy product id for the HAT.

throttled:
Output of "vcgencmd get_throttled" shows any power/thermal issues on the controller.

hat_alert:
Status of the ALERT LED (0=off, 1=on).

pX:
Status of node pX power (0=off, 1=on).

uX: [only shown on Cluster CTRL devices]
Status of usbboot for node pX (0=off, 1=on).

ctrlX: [only shown for Cluster CTRL devices]
Space separated fields for FW: firmware version and where supported ADCx: voltages, (inaccurate) Tx: temperature, etc.

Author: Chris Burton
Last update: 2020-11-29


How are nodes numbered with Cluster CTRL/Cluster HAT (resolve order conflict)?

The pX number of a node for control is set by it's priority (this might not be the same as the hostname/configuration of the image/usbboot filesystem).

Any detected Cluster HAT (v1.x or v2.x) is given highest priority so this will always be p1-p4 if it's detected.

All Cluster CTRL devices (Triple/A+6/etc.) have an "order" stored in EEPROM/FLASH. By default the order is set to 20 for the Triple and 10 for the A+6. The lower this "order" number the higher the priority.

So if you have ClusterCTRL Triple devices with order 20 and 21 the node numbers will be as follows.

With Cluster HAT
p1-p4 ClusterHAT
p5-p7 Triple (priority 20)
p8-p10 Triple (priority 21)

Without Cluster HAT
p1-p3 Triple (priority 20)
p4-p6 Triple (priority 21)

If you have more than one ClusterCTRL device with the same "order" the "clusterctrl" tool will show an error, you would need to remove all but one of the conflicting ClusterCTRL devices to set a new order.

$ clusterctrl status
ERROR: Duplicate ClusterCTRL 'order' found
I2C Bus: 4 Order: 20
I2C Bus: 3 Order: 20

Changing ClusterCTRL devices order

If you have a conflict first unplug all but one device with the same order.

To  change the order stored of a ClusterCTRL device you need the existing order number which can be seen using "clusterctrl status".

To temporarily change the order of a Cluster CTRL device use "clusterctrl setorder <old order> <new order>", so to change from order 20 to order 21 run.

clusterctrl setorder 20 21

The order can be set between 1 and 255.

The order is now set temporarily and will revert back to the old order after a power cycle or reset. To make the change perminant the settings must be saved to EEPROM.

clusterctrl save 21

The new order (and usbboot/power) will now be saved to the EEPROM and used at next power on.

Author: Chris Burton
Last update: 2019-08-08


How do I write new firmware to Cluster Triple?

Control of the Cluster Triple is done using an ATmega328P (or ATmega328PB), the IC can be reprogrammed using either a standard ISCP programmer or you can also use a Raspberry Pi.  If you're a regular programming AVR then you can skip most of this document but please remember NOT to alter the Fuses as you may end up "bricking" the IC requiring it to be removed and reprogammed with a parallel programmer.

This guide explains how to use a Raspberry Pi to write updated firmware to the Cluster Triple. 

To program the Cluster Triple following this guide you'll need a Raspberry Pi running Raspbian (or Raspbian based distribution) with a 40 pin header, five jumper wires (female-female) and power supplies for both Pi and Cluster Triple, etc.

Connect the wires between Cluster Triple and Raspberry Pi  as shown in the table below.

Cluster Triple Raspbrrry Pi (physical/ BCM)
1 / MISO 12 / BCM18
2 / 3v3 No connect
3 / SCK 18 / BCM24
4 / MOSI 16 / BCM23
5 / RST 32 / BCM12
6 / GND 6 / GND

The physical pins on the Cluster Triple and Raspberry Pi can be see below (Pi GPIO pinout from pinout.xyz).

Cluster Triple PGM Pinout

Raspberry Pi Pinout

Once connected up you should have something like this.

Program Cluster Triple wiring

Program Cluster Triple wiring

 Once you have the wiring setup between the Pi and Cluster Triple  ensure all Compute modules have been removed from the Triple, power the Cluster Triple via the "PWR" port, remove any cables from the "USB" port on the Triple, power on the Pi.

Log into the Pi and install the required software Git (to download the firmware) and Avrdude (programming).

sudo apt-get -y install git avrdude

Download the ClusterCTRL firmware.

git clone https://github.com/burtyb/ClusterCTRL.git

If you haven't added the programmers used by ClusterCTRL to the avrdude.conf file previously these can be added by running.

sudo bash -c "cat ClusterCTRL/clustertriple-1.0/avrdude.conf >> /etc/avrdude.conf"

You can now either build your own firmware or install prebuild firmware.

Installing prebuilt firmware

To see the currently available firmware files.

ls ClusterCTRL/firmware/

Files are numbered with the version at the start, then the board type and harware version. You MUST pick the firmware to match the hardware you have, for example 1.2_clustertriple-1.0.hex for the Cluster Triple v1.0. Installing the wrong firmware can brick the ClusterCTRL device. If in doubt please ask on the forum first.

Double check the wiring and once you're ready program the AVR (replacing FIRMWARE with the .hex firmware file to be written "...flash:w:ClusterCTRL/firmware/1.2_clustertriple-1.0.hex" for example).

sudo avrdude -D -c pi_1 -p atmega328p -U flash:w:ClusterCTRL/firmware/FIRMWARE

If you see a Device Signature "avrdude: Device signature = 0x1e9516 (probably m328pb)", then your Cluster Triple has an ATmega328pb and needs to be programmed using a different command.

sudo avrdude -D -c pi_1 -p atmega328pb -U flash:w:ClusterCTRL/firmware/FIRMWARE

 You should then see the AVR being programmed, after a few seconds you should see "avrdude done. Thank you.". You can now unplug the PWR cable to the Cluster Triple, remove the jumper wires and then, replace the Compute Modules and connect up as normal.

Building and installing firmware

Install the software to compile firmware for the AVR

apt-get install gcc-avr avr-libc

Change into the directory of the ClusterCTRL devices you have.

cd ClusterCTRL/clustertriple-1.0/

Build the firmware using make.

make

You should then see the firmware being built and once complete it should check the size of the firmware file - something similar to this (the numbers may be different).

...
...
avr-objcopy -j .text -j .data -O ihex firmware.bin firmware.hex
../src/checksize firmware.bin 32768 2048
ROM: 7450 bytes (data=74)
RAM: 144 bytes

If the firmware has been built correctly you will need to change to the root user and change back to the build directory.

sudo -i
cd ~pi/ClusterCTRL/clustertriple-1.0/

You can now program the IC

make program

 The IC is then detected (ATmega328p/ATmega328pb) and programmed, if you see any errors please check/post on the forum.

Type exit to get back to the "pi" user.

exit

You can now unplug the PWR cable to the Cluster Triple, remove the jumper wires and then, replace the Compute Modules and connect and power up as normal.

Author: Chris Burton
Last update: 2019-08-29


How do I enable autologin on TTY running on the Controllers serial port [Raspbian Buster/Bullseye/Bookworm]?

To enable auto logins on the TTY running on any serial port on the Controller log in and run the following commands changing "USERNAME" to your username and reboot.

sudo mkdir -p /etc/systemd/system/serial-getty@ttyS0.service.d/
sudo bash -c "cat > /etc/systemd/system/serial-getty@ttyS0.service.d/autologin.conf << EOF
[Service]
ExecStart=
ExecStart=-/sbin/agetty --autologin USERNAME --noclear %I \$TERM
EOF"

And to disable it run the following command and reboot.

sudo rm -rf /etc/systemd/system/serial-getty@ttyS0.service.d/

When enabling this all users with access to the Controller Pi serial port will be able to log in without any authentication.

Author: Chris Burton
Last update: 2024-01-02


How do I tell a Raspberry Pi to ignore a HAT EEPROM?

To prevent loading of device tree files (etc) from a HAT EEPROM add the following to the config.txt file in the /boot partition.

force_eeprom_read=0

Author: Chris Burton
Last update: 2019-10-11


Troubleshooting missing Pi Zeros in a ClusterHAT

If you've got the Cluster HAT connected and you're having problems with one or more Pi Zeros not being visible after booting with an SD card please check through these steps and report any issues on the forum.

 

1) After powering on the Controller Pi is the PWR LED on the HAT lit up, and after Raspbian boots does the ALERT LED go from dim to off?


- If not check the HAT is plugged into the Controller Pi properly.

2) When you run "clusterctrl status" does it say "clusterhat:1" ?

- If it doesn't check the Cluster HAT is pressed onto the Controller Pi GPIO pins properly (you will need to reboot the Controller Pi before it will be detected).

3) Check the USB hub is visible, run "clusterctrl hub off", "lsusb -t" (save the output), "clusterctrl hub on", run "lsusb -t" again and compare.


- If you don't see the 4 port HUB check the USB cable (try another cable ensuring the cable supports data as some are power only).

4) With the power off to the Pi Zeros "clusterctrl off", remove their SD cards, run "sudo service clusterctrl-rpiboot stop", "clusterctrl on" and then wait a minute.

5) Do the P1-P4 LED (on the Cluser HAT) come on (you may need to run "clusterctrl led on" if you've turned the LED off)?


- If not then probably an issue with the HAT.

6) Do you see four 12M USB devices hanging off the USB hub detected above?


- If you do it's a problem with either the SD card itself or the Raspbian installation on the SD cards, please try to write the image again and/or use a different SD card if you have one available.
- If you don't them make sure you've pressed the Pi Zeros onto the connectors far enough (if new they will normally click), you should be able to see the hooks through the holes in the connectors on the Pi Zero if it's plugged in far enough.

 

7) Power off the Pi Zeros "clusterctrl off", replace the SD cards, and power on "clusterctrl on" and wait a minute or two (the initial boot will take longer as it will resize the image to the size of the SD card).

8) Does "ls -l /dev/ttyACM* /dev/ttypi*" show the USB Gadget Serial ports and links?

9) Does "sudo ifconfig -a" show the ethpi1 to ethpi4 interfaces?

- If not, do you have any "renameX" interfaces? If so then it's likely you have a MAC address conflict (the MAC should correspond to the Px number so P1 should use "00:22:82:ff:fe:01", P2 "00:22:82:ff:fe:02", etc.). If you have a "renameX" interface check if the MAC address is the same as one of the ethpiX interfaces, if so you will need to either reconfigure the SD card (see Intermediate Setup section) or re-image the SD card with the correct image.

 

 If you're still having problems please post details of your setup to the forum - Cluster HAT model, Controller Pi Model, filenames of the image used on the controller (cnat/cbridge at a minimum) and Pi Zeros image filenames,  how you're connecting to the internet (wifi/ethernet/etc) - basically please explain it like me like I don't know your setup :).

Author: Chris Burton
Last update: 2020-03-03


What are the differences in Cluster HAT versions?

Cluster HAT v1.0

USB connector: On left (SMT)
Serial connector: TH
GPIO connector: Bottom entry + header
Power connector: 2 pin header (0.1")
Solder jumpers top: W_EN (Write protect EEPROM) / PWR (Power source USB or GPIO)
Solder jumpers bottom: None
USB Hub control: None

Cluster HAT v1.1

USB connector: On left (SMT+TH)
Serial connector: SMT
GPIO connector: Header
Power connector: None
Solder jumpers top: W_EN (Write protect EEPROM) / PWR (Power source USB or GPIO)
Solder jumpers bottom: None
USB Hub control: None

Cluster HAT v1.2

USB connector: On left (SMT+TH)
Serial connector: SMT
GPIO connector: Header
Power connector: None
Solder jumpers top: PWR (Power source USB or GPIO)
Solder jumpers bottom: None
USB Hub control: None

Cluster HAT v1.3

USB connector: On left (SMT+TH)
Serial connector: SMT
GPIO connector: Header
Power connector: SMT (underside)
Solder jumpers top: Bigger PWR (Power source USB or GPIO)
Solder jumpers bottom: None
USB Hub control: None

Cluster HAT v2.0

USB connector: Front (SMT+TH)
Serial connector: SMT
GPIO connector: Header
Power connector: SMT (underside)
Solder jumpers top: Bigger PWR (Power source USB or GPIO)
Solder jumpers bottom: WP (EEPROM Write protect - bridge to prevent writes) / POS (Power on state - cut to auto power on)
USB Hub control: Power
Pi4 Controller Reboot: Rebooting Pi4 controller will reset I2C expander powering off Pi Zeros (use 3v3 Sustain to fix)

Cluster HAT v2.1

Inrush current issue, not released.

Cluster HAT v2.2

USB connector: Front (SMT+TH)
Serial connector: SMT
GPIO connector: Header
Power connector: SMT (underside) / Front header for 3.5mm screw terminal
Solder jumpers top: Bigger PWR (GPIO or GPIO)
Solder jumpers bottom: WP (EEPROM Write protect - bridge to prevent writes) / POS (Power on state - cut to auto power on)
USB Hub control: Reset
Pi4 Controller Reboot: Rebooting Pi4 controller will reset I2C expander powering off Pi Zeros (use 3v3 Sustain to fix)

Cluster HAT v2.3

USB connector: Front (SMT+TH)
Serial connector: SMT
GPIO connector: Header
Power connector: SMT (underside) / Front header for 3.5mm screw terminal
Solder jumpers top: Bigger PWR (GPIO or GPIO)
Solder jumpers bottom: WP (EEPROM Write protect - bridge to prevent writes) / POS (Power on state - cut to auto power on)
USB Hub control: Reset
Pi4 Controller Reboot: Rebooting Pi4 controller will reset I2C expander powering off Pi Zeros (use 3v3 Sustain to fix)

Cluster HAT v2.4

USB connector: Front (SMT+TH)
Serial connector: SMT
GPIO connector: Header
Power connector: SMT (underside) / Front header for 3.5mm screw terminal
Solder jumpers top: Bigger PWR (GPIO or GPIO)
Solder jumpers bottom: WP (EEPROM Write protect - bridge to prevent writes) / POS (Power on state - cut to auto power on) / XA0 & XA1 I/O Expander I2C Address / EA0 & EA1 HAT EEPROM I2C Address
USB Hub control: Reset
Pi4 Controller Reboot: No issue.

Cluster HAT v2.5

USB connector: Front (SMT+TH)
Serial connector: SMT
GPIO connector: Header
Power connector: SMT (underside) / Front header for 3.5mm screw terminal
Fan connector: 5V 2-pin 2.54mm pitch unpopulated header (+5V closest to side of board) 200mA max.
Solder jumpers top: Bigger PWR (GPIO or GPIO)
Solder jumpers bottom: WP (EEPROM Write protect - bridge to prevent writes) / POS (Power on state - cut to auto power on) / XA0 & XA1 - I/O Expander I2C Address / EA0 & EA1 - HAT EEPROM I2C Address / ID_SDA & ID_SCL - HAT EEPROM cut both to disconnect
USB Hub control: Reset
Pi4 Controller Reboot: No issue.

Key: TH = Through Hole / SMT = Surface Mount

Author: Chris Burton
Last update: 2023-02-05


How do I use the solder jumpers on the Cluster HAT?

Solder jumpers on the Cluster HAT allow you to modify the default behaviour and configuration. To change the configuration you will need to either cut the link between the pads and/or bridge the pads with solder depending on their default state (open/closed).

PWR - Power source

This solder jumper is present on all Cluster HAT versions.

- By default the link (either soldered or via cuttable trace on the PCB) is between the centre pad and "RPi" which takes power for the Pi Zeros from the GPIO header (USB hub IC is always powered via the USB cable).
- To power the Pi Zeros from the USB cable you will need to disconnect the link (either remove the solder bridge or by cutting the PCB trace) between RPi and the centre pad and then bridge the centre pad to the side labelled "USB". To do this you will need to use your own USB cable capable of carrying the high current required for the Pi Zeros (not supplied).
- To power the Pi Zeros from either the SMD pads on the underside of the Cluster HAT or the (unpopulated) 3.5mm pitch screw terminals at the front you will need to ensure the centre pad is disconnected from both the "RPi" and "USB" pads.

WP - (HAT EEPROM) Write Protect

Cluster HAT v1.0/1.1 - Close the solder jumper (default) to prevent writing to HAT EEPROM, remove the solder bridge to allow writes.
Cluster HAT v2.x - Solder jumper should be open (default) to allow writes to HAT EEPROM, to prevent writes bridge the solder jumper.

POS - Power on state

This jumper allows power to the Pi Zeros to be enabled by default when power is applied to the Cluster HAT in hardware.

Cluster HAT v2.x
- To leave the Pi Zeros powered off until commanded by software the link should be open (default).
- To power on the Pi Zeros as soon as power is applied bridge the solder jumper.

XA0/XA1 - I/O Expander I2C address

To allow multiple Cluster HAT to be used on the same Controller Pi (using the Butterfly board for example) the I2C address of the I/O Expanders must be unique. We strongly advise updating the HAT EEPROM I2C address in line with the Expander address.

Cluster HAT v2.4/5
- The I2C address is set by pulling the pins address pins XA0/XA1 HIGH or LOW.
- The solder jumpers are 2-way. The pad closest to the GPIO header is HIGH (3.3v) and the pad furthest away is LOW (GND), the centre pin is connected to the address line.
- By default the 2-way jumpers are both set LOW using a trace on the board. To change XA0 or XA1 to HIGH you must first cut the trace between the centre pad and GND pad (furthest away from the GPIO header) and then bridge between the centre pad and HIGH pad (closest to GPIO header).

XA1 XA0 I2C Address
LOW LOW 0x20 (default)
LOW HIGH 0x21
HIGH LOW 0x22
HIGH HIGH 0x23

 

Updates to the clusterctrl software are in progress but currently to control multiple Cluster HAT you can clone the existing clusterctrl tool.

For example to control a Cluster HAT on address 0x23 you can do the following.

sudo cp /sbin/clusterctrl /sbin/clusterctrl23
sudo cp /etc/default/clusterctrl /etc/default/clusterctrl23
sudo sed -i "s#0x20#0x23#" /sbin/clusterctrl23
sudo sed -i "s#default/clusterctrl#default/clusterctrl23#" /sbin/clusterctrl23

You should now be able to control the additional Cluster HAT using "clusterctrl23 on" etc.

EA0/EA1 - HAT EEPROM I2C address

To allow multiple Cluster HAT to be used on the same Controller Pi (using the Butterfly board for example) the I2C address of the HAT EEPROM need to be unique (or disconnceted see ID_* below). We strongly advise updating the I/O Expander I2C address in line with the HAT EEPROM address.

Cluster HAT v2.4/5
- The I2C address can be set by opening / closing (bridging with solder) the solder jumpers as follows.

EA1 EA0 I2C Address
Open Open 0x50 (default)
Open Closed 0x51
Closed Open 0x52
Closed Closed 0x53

 

ID_SCL / ID_SDA - HAT EEPROM disconnect

Cluster HAT v2.5 - When using multiple HAT with a Pi it can be useful to be able to disconnect the HAT EEPROM to prevent conflicts, these solder jumpers offer this option.

- To connect the HAT EEPROM to the ID pins the solder jumper should be linked (default).
- To remove the HAT EEPROM from the HAT I2C bus cut BOTH solder jumpers.

FAN - Disconnect fan MOSFET

Cluster HAT v2.5 - If you're using GPIO BCM18 for something else you may want to disconnect it from the MOSFET.
- To connect the FAN MOSFET to the GPIO line and enable the fan header to power a fan the link should be bridged (default).
- To disconnect the MOSFET from the GPIO pin cut the solder jumper.

Author: Chris Burton
Last update: 2024-02-03


How do I use the FAN header on Cluster HAT v2.5?

The FAN header on the Cluster HAT v2.5 is controlled via GPIO pin BCM18 (set to output and set high to turn the fan on, and low to turn the fan off).

To connect a FAN to the Cluster HAT first solder a 2 pin 2.54 mm (0.1") pitch header onto the FAN pads, the +5V side is closest to the side of the Cluster HAT PCB.

To control the FAN based on temperature of the Controller you can simply load the gpio-fan Device Tree overlay by adding the following to the bottom of your /boot/config.txt file (you can change the 75000 which relates to 75C when the fan will turn on, once cooled 10C less than this temperature the fan will turn off).

dtoverlay=gpio-fan,gpiopin=18,temp=75000

Please note the 5V FAN should have an integral flyback diode (most small fans already do) and not use more than 200mA.

Author: Chris Burton
Last update: 2020-07-02


How do I manually install clusterctrl?

If you're using one of the provided ClusterCTRL images the cluserctrl tool will already be installed but those using the ClusterCTRL Stack or only want to manage power for a ClusterHAT for example may want to install it manually.

To install the tool manually you can normally run through the following steps as the root user (you can usually switch to the root user by using "sudo -i").

TMPINSTALL=/tmp/clusterhat
apt update
apt install git libusb-1.0-0-dev python3-usb python3-libusb1 python3-smbus python-is-python3 # When using Python3
# apt install git libusb-1.0-0-dev python-usb python-libusb1 python-smbus # When using older installs with Python2
git clone https://github.com/burtyb/clusterhat-image.git $TMPINSTALL
mkdir /usr/share/clusterctrl/
cp $TMPINSTALL/files/usr/sbin/cluster* /usr/sbin/
cp $TMPINSTALL/files/usr/share/clusterctrl/default-clusterctrl /etc/default/clusterctrl
cp $TMPINSTALL/files/etc/udev/rules.d/90-clusterctrl.rules /etc/udev/rules.d/
cp $TMPINSTALL/files/usr/lib/systemd/system/clusterctrl-init.service /usr/lib/systemd/system/
cp -r $TMPINSTALL/files/usr/share/clusterctrl/python/ /usr/share/clusterctrl/
echo 'TYPE=c' >> /etc/default/clusterctrl
udevadm control --reload-rules
systemctl enable clusterctrl-init
raspi-config nonint do_i2c 0 # Enable I2C
rm -rf $TMPINSTALL

If you see an error on any of these stages please create a new thread on the support forum or open a support ticket with details of the distribution you're using and the issue you're having.

Now disconnect and reconnect the USB cable (if using a ClusterCTRL device) and then check it's found the device.

clusterctrl status

This should then find and show the status of either a ClusterHAT and/or any ClusterCTRL devices you have connected.

Author: Chris Burton
Last update: 2023-11-26


Can the Pi Zero 2 W be used in the Cluster HAT?

For light/average use the Cluster HAT should work OK with four of the newer Pi Zero 2 W, the main thing to be aware of is the potential power consumption.

We've taken the following measurements with a Ethernet connected Pi 4.

Idle Pi 4 ~2.4 Watts and ~5.8 Watts running stress.
Idle Pi Zero 2 W ~1.3 Watts and ~3.4 Watts running stress.
Idle Pi Zero ~1.3 Watts and ~1.6 Watts running stress.

Which means a Pi4 and 4xPi02W could use between ~7.2 Watts when idle and ~19.4 Watts with all 5 machines running stress.

The 19.4 Watts is higher than the official Pi 4 Power supplies rating of 15.3 Watts (vs the Pi4+4*Pi0 ~7.6 to ~12.2 Watts).
For those running lighter or non-concurrent loading - say a Pi4 running stress, two idle Pi02W and two Pi02W running stress the power usage (15.2 Watts) fits within the rating.

Please remember peripherals such as USB Flash drives, Cameras, Mouse, Keyboards, using WiFi, overclocking, etc. will all require more power.

Author: Chris Burton
Last update: 2021-12-13


How do I disable automount in Raspberry Pi OS (Bullseye with GUI)?

Create a file ~/.config/pcmanfm/LXDE-pi/pcmanfm.conf for the "default" user (the one created when installed) with the following contents.

[volume]
mount_on_startup=0
mount_removable=0
autorun=1

After a reboot USB disks/etc. will not be automatically mounted anymore.

Author: Chris Burton
Last update: 2023-07-16


How do I write new firmware to Cluster Stack?

Control of the ClusterCTRL Stackis done using an ATmega328P (or ATmega328PB), the IC can be reprogrammed using either a standard ISCP programmer or you can also use a Raspberry Pi.  If you're a regular programming AVR then you can skip most of this document but please remember NOT to alter the Fuses as you may end up "bricking" the IC requiring it to be removed and reprogammed with a parallel programmer.

This guide explains how to use a Raspberry Pi to write updated firmware to the ClusterCTRL Stack. 

To program the ClusterCTRL Stack following this guide you'll need a Raspberry Pi running Raspberry Pi OS with a 40 pin header, five jumper wires (female-female) and power supplies for both Pi and ClusterCTRL Stack, etc.

Connect the wires between Cluster Triple and Raspberry Pi  as shown in the table below.

ClusterCTRL Stack Raspbrrry Pi (physical/ BCM)
1 / MISO 12 / BCM18
2 / 3v3 No connect
3 / SCK 18 / BCM24
4 / MOSI 16 / BCM23
5 / RST 32 / BCM12
6 / GND 6 / GND

The physical pins on the ClusterCTRL Stack and Raspberry Pi can be see below (Pi GPIO pinout from pinout.xyz).

Cluster Stack PGM Pinout

Raspberry Pi Pinout

Once connected up you should have something like this.

Program ClusterCTRK Stack wiring

Program ClusterCTRL Stack wiring

Once you have the wiring setup between the Pi and Cluster Triple  ensure either the backplane is disconnected from the ClusterCTRL Controller, power the ClusterCTRL Stack via the barell jack, remove any cables from the "USB" port on the ClusterCTRL Controller and power on the Pi.

Log into the Pi and install the required software Git (to download the firmware) and Avrdude (programming).

sudo apt update
sudo apt -y install git avrdude

Download the ClusterCTRL firmware.

git clone https://github.com/burtyb/ClusterCTRL.git

If you haven't added the programmers used by ClusterCTRL to the avrdude.conf file previously these can be added by running.

sudo bash -c "cat ClusterCTRL/clustertriple-1.0/avrdude.conf >> /etc/avrdude.conf"

You can now either build your own firmware or install prebuild firmware.

Installing prebuilt firmware

To see the currently available firmware files.

ls ClusterCTRL/firmware/

Files are numbered with the version at the start, then the board type and harware version. You MUST pick the firmware to match the hardware you have, for example 1.2_clusterstack-1.0.hex for the ClusterCTRL Stack v1.0. Installing the wrong firmware can brick the ClusterCTRL device. If in doubt please ask on the forum first.

Double check the wiring and once you're ready program the AVR (replacing FIRMWARE with the .hex firmware file to be written "...flash:w:ClusterCTRL/firmware/1.2_clusterstack-1.0.hex" for example).

sudo avrdude -c pi_1 -p atmega328p -U flash:w:ClusterCTRL/firmware/FIRMWARE

If you see a Device Signature "avrdude: Device signature = 0x1e9516 (probably m328pb)", then your ClusterCTRL Stack has an ATmega328pb and needs to be programmed using a different command.

sudo avrdude -c pi_1 -p atmega328pb -U flash:w:ClusterCTRL/firmware/FIRMWARE

 You should then see the AVR being programmed, after a few seconds you should see "avrdude done. Thank you.". You can now unplug the PWR cable to the Cluster Triple, remove the jumper wires and then, replace the Compute Modules and connect up as normal.

Building and installing firmware

Install the software to compile firmware for the AVR

sudo apt -y install gcc-avr avr-libc

Change into the directory of the ClusterCTRL devices you have.

cd ClusterCTRL/clusterstack-1.0/

You can now modify the firmware as required, for example to increase the delay between powering on the nodes from 1 second to 5 seconds edit config.h and change the value for AUTOONDELAY from 1000 to 5000 (the delay is in milliseconds).

Build the firmware using make.

make

You should then see the firmware being built and once complete it should check the size of the firmware file - something similar to this (the numbers may be different).

...
...
avr-objcopy -j .text -j .data -O ihex firmware.bin firmware.hex
../src/checksize firmware.bin 32768 2048
ROM: 7304 bytes (data=26)
RAM: 96 bytes

If the firmware has been built correctly you can now program the IC.

sudo make program

 The IC is then detected (ATmega328p/ATmega328pb) and programmed, if you see any errors please check/post on the forum.

You can now unplug the PWR cable to the ClusterCTRL Stack, remove the jumper wires and then,connect back up as normal.

Author: Chris Burton
Last update: 2023-07-25


How do I write new firmware to Cluster Triple

Control of the Cluster Triple is done using an ATmega328P (or ATmega328PB), the IC can be reprogrammed using either a standard ISCP programmer or you can also use a Raspberry Pi.  If you're a regular programming AVR then you can skip most of this document but please remember NOT to alter the Fuses as you may end up "bricking" the IC requiring it to be removed and reprogammed with a parallel programmer.

This guide explains how to use a Raspberry Pi to write updated firmware to the Cluster Triple. 

To program the Cluster Triple following this guide you'll need a Raspberry Pi running Raspbian (or Raspbian based distribution) with a 40 pin header, five jumper wires (female-female) and power supplies for both Pi and Cluster Triple, etc.

Connect the wires between Cluster Triple and Raspberry Pi  as shown in the table below.

Cluster Triple Raspbrrry Pi (physical/ BCM)
1 / MISO 12 / BCM18
2 / 3v3 No connect
3 / SCK 18 / BCM24
4 / MOSI 16 / BCM23
5 / RST 32 / BCM12
6 / GND 6 / GND

The physical pins on the Cluster Triple and Raspberry Pi can be see below (Pi GPIO pinout from pinout.xyz).

Cluster Triple PGM Pinout

Raspberry Pi Pinout

Once connected up you should have something like this.

Program Cluster Triple wiring

Program Cluster Triple wiring

 Once you have the wiring setup between the Pi and Cluster Triple  ensure all Compute modules have been removed from the Triple, power the Cluster Triple via the "PWR" port, remove any cables from the "USB" port on the Triple, power on the Pi.

Log into the Pi and install the required software Git (to download the firmware) and Avrdude (programming).

sudo apt-get -y install git avrdude

Download the ClusterCTRL firmware.

git clone https://github.com/burtyb/ClusterCTRL.git

If you haven't added the programmers used by ClusterCTRL to the avrdude.conf file previously these can be added by running.

sudo bash -c "cat ClusterCTRL/clustertriple-1.0/avrdude.conf >> /etc/avrdude.conf"

You can now either build your own firmware or install prebuild firmware.

Installing prebuilt firmware

To see the currently available firmware files.

ls ClusterCTRL/firmware/

Files are numbered with the version at the start, then the board type and harware version. You MUST pick the firmware to match the hardware you have, for example 1.2_clustertriple-1.0.hex for the Cluster Triple v1.0. Installing the wrong firmware can brick the ClusterCTRL device. If in doubt please ask on the forum first.

Double check the wiring and once you're ready program the AVR (replacing FIRMWARE with the .hex firmware file to be written "...flash:w:ClusterCTRL/firmware/1.2_clustertriple-1.0.hex" for example).

sudo avrdude -D -c pi_1 -p atmega328p -U flash:w:ClusterCTRL/firmware/FIRMWARE

If you see a Device Signature "avrdude: Device signature = 0x1e9516 (probably m328pb)", then your Cluster Triple has an ATmega328pb and needs to be programmed using a different command.

sudo avrdude -D -c pi_1 -p atmega328pb -U flash:w:ClusterCTRL/firmware/FIRMWARE

 You should then see the AVR being programmed, after a few seconds you should see "avrdude done. Thank you.". You can now unplug the PWR cable to the Cluster Triple, remove the jumper wires and then, replace the Compute Modules and connect up as normal.

Building and installing firmware

Install the software to compile firmware for the AVR

apt-get install gcc-avr avr-libc

Change into the directory of the ClusterCTRL devices you have.

cd ClusterCTRL/clustertriple-1.0/

Build the firmware using make.

make

You should then see the firmware being built and once complete it should check the size of the firmware file - something similar to this (the numbers may be different).

...
...
avr-objcopy -j .text -j .data -O ihex firmware.bin firmware.hex
../src/checksize firmware.bin 32768 2048
ROM: 7450 bytes (data=74)
RAM: 144 bytes

If the firmware has been built correctly you will need to change to the root user and change back to the build directory.

sudo -i
cd ~pi/ClusterCTRL/clustertriple-1.0/

You can now program the IC

make program

 The IC is then detected (ATmega328p/ATmega328pb) and programmed, if you see any errors please check/post on the forum.

Type exit to get back to the "pi" user.

exit

You can now unplug the PWR cable to the Cluster Triple, remove the jumper wires and then, replace the Compute Modules and connect and power up as normal.

Author: Chris Burton
Last update: 2023-07-25


How do I write new firmware to Cluster Stack

Control of the ClusterCTRL Stackis done using an ATmega328P (or ATmega328PB), the IC can be reprogrammed using either a standard ISCP programmer or you can also use a Raspberry Pi.  If you're a regular programming AVR then you can skip most of this document but please remember NOT to alter the Fuses as you may end up "bricking" the IC requiring it to be removed and reprogammed with a parallel programmer.

This guide explains how to use a Raspberry Pi to write updated firmware to the ClusterCTRL Stack. 

To program the ClusterCTRL Stack following this guide you'll need a Raspberry Pi running Raspberry Pi OS with a 40 pin header, five jumper wires (female-female) and power supplies for both Pi and ClusterCTRL Stack, etc.

Connect the wires between Cluster Triple and Raspberry Pi  as shown in the table below.

ClusterCTRL Stack Raspbrrry Pi (physical/ BCM)
1 / MISO 12 / BCM18
2 / 3v3 No connect
3 / SCK 18 / BCM24
4 / MOSI 16 / BCM23
5 / RST 32 / BCM12
6 / GND 6 / GND

The physical pins on the ClusterCTRL Stack and Raspberry Pi can be see below (Pi GPIO pinout from pinout.xyz).

Cluster Stack PGM Pinout

Raspberry Pi Pinout

Once connected up you should have something like this.

Program ClusterCTRK Stack wiring

Program ClusterCTRL Stack wiring

Once you have the wiring setup between the Pi and Cluster Triple  ensure either the backplane is disconnected from the ClusterCTRL Controller, power the ClusterCTRL Stack via the barell jack, remove any cables from the "USB" port on the ClusterCTRL Controller and power on the Pi.

Log into the Pi and install the required software Git (to download the firmware) and Avrdude (programming).

sudo apt update
sudo apt -y install git avrdude

Download the ClusterCTRL firmware.

git clone https://github.com/burtyb/ClusterCTRL.git

If you haven't added the programmers used by ClusterCTRL to the avrdude.conf file previously these can be added by running.

sudo bash -c "cat ClusterCTRL/clustertriple-1.0/avrdude.conf >> /etc/avrdude.conf"

You can now either build your own firmware or install prebuild firmware.

Installing prebuilt firmware

To see the currently available firmware files.

ls ClusterCTRL/firmware/

Files are numbered with the version at the start, then the board type and harware version. You MUST pick the firmware to match the hardware you have, for example 1.2_clusterstack-1.0.hex for the ClusterCTRL Stack v1.0. Installing the wrong firmware can brick the ClusterCTRL device. If in doubt please ask on the forum first.

Double check the wiring and once you're ready program the AVR (replacing FIRMWARE with the .hex firmware file to be written "...flash:w:ClusterCTRL/firmware/1.2_clusterstack-1.0.hex" for example).

sudo avrdude -c pi_1 -p atmega328p -U flash:w:ClusterCTRL/firmware/FIRMWARE

If you see a Device Signature "avrdude: Device signature = 0x1e9516 (probably m328pb)", then your ClusterCTRL Stack has an ATmega328pb and needs to be programmed using a different command.

sudo avrdude -c pi_1 -p atmega328pb -U flash:w:ClusterCTRL/firmware/FIRMWARE

 You should then see the AVR being programmed, after a few seconds you should see "avrdude done. Thank you.". You can now unplug the PWR cable to the Cluster Triple, remove the jumper wires and then, replace the Compute Modules and connect up as normal.

Building and installing firmware

Install the software to compile firmware for the AVR

sudo apt -y install gcc-avr avr-libc

Change into the directory of the ClusterCTRL devices you have.

cd ClusterCTRL/clusterstack-1.0/

You can now modify the firmware as required, for example to increase the delay between powering on the nodes from 1 second to 5 seconds edit config.h and change the value for AUTOONDELAY from 1000 to 5000 (the delay is in milliseconds).

Build the firmware using make.

make

You should then see the firmware being built and once complete it should check the size of the firmware file - something similar to this (the numbers may be different).

...
...
avr-objcopy -j .text -j .data -O ihex firmware.bin firmware.hex
../src/checksize firmware.bin 32768 2048
ROM: 7304 bytes (data=26)
RAM: 96 bytes

If the firmware has been built correctly you can now program the IC.

sudo make program

 The IC is then detected (ATmega328p/ATmega328pb) and programmed, if you see any errors please check/post on the forum.

You can now unplug the PWR cable to the ClusterCTRL Stack, remove the jumper wires and then,connect back up as normal.

Author: Chris Burton
Last update: 2023-07-25