October 2013 – 工作心得

讓VMware ESXi 5.0 正確辨識SSD及啟用host cache

參考資料:

http://www.yellow-bricks.com/2011/08/18/swap-to-host-cache-aka-swap-to-ssd/

http://www.virtuallyghetto.com/2011/07/how-to-trick-esxi-5-in-seeing-ssd.html

http://www.informationweek.com/the-vmware-write-cache-challenge-solved/240147594

環境:

Dell R610 300G sas x4 (RAID5), 200G sata ssd x 2 (RAID1)

裝了2顆SSD但是在vCenter上無法正常辨識為SSD，因此無法啟用host cache，必須利用下面的選項將SSD datastore標記為SSD

指令備忘

esxcli storage nmp satp rule add --satp VMW_SATP_LOCAL --device naa.6782bcb01ef0cf001a00e4a3722e6b00 --option=enable_ssd

esxcli storage nmp satp rule list | grep enable_ssd

esxcli storage core claiming reclaim -d naa.6782bcb01ef0cf001a00e4a3722e6b00

esxcli storage core device list --device=naa.6782bcb01ef0cf001a00e4a3722e6b00

vim-cmd hostsvc/storage/refresh

todo:

host cache是write or read cache?(待確認)

從Swapping to Host Cache看起來是VM swap file的write cache

從IP清單轉fortinet firewall語法

<?php

function gen_fw_config_mulit($IP){
  #10.28.101.135-155
  $tmp = preg_split('/\./', $IP);
  $ip_head = "{$tmp[0]}.{$tmp[1]}.{$tmp[2]}";
  $tmp2 = preg_split('/-/', $tmp[3]);
  $ip_start = $tmp2[0];
  $ip_end = $tmp2[1];

  #output config
  echo "edit \"$ip_head.[$ip_start-$ip_end]\"\n";
  echo "    set associated-interface \"P1\"\n";
  echo "    set type iprange\n";
  echo "    set end-ip $ip_head.$ip_end\n";
  echo "    set start-ip $ip_head.$ip_start\n";
  echo "next\n";
}

function gen_fw_config_single($IP){
  #10.28.101.135
  #output config
  echo "edit \"$IP/32\"\n";
  echo "    set associated-interface \"P1\"\n";
  echo "    set subnet $IP 255.255.255.255\n";
  echo "next\n";
}

?>

<?php

$ip_file = 'iplist.txt';

require_once 'config.inc.php';
$lines = file($ip_file);
foreach($lines as $line){
  $line = trim($line);
  if (stripos($line, '-') !== false){ //multi IP
    #echo "#$line\n";
    gen_fw_config_mulit($line);
  }else { //single IP
    gen_fw_config_single($line);
  }//if
}//foreach
?>

VMware ESXi 5.0 Enable SNMP

install Vsphere CLI first

#check setting
vicfg-snmp.pl --server 172.30.65.240 --username root --password xxxxxxxx --show

#set snmp community
vicfg-snmp.pl --server 172.30.65.240 --username root --password xxxxxxxx --communities public

#enable snmp service
vicfg-snmp.pl --server 172.30.65.240 --username root --password xxxxxxxx --enable

使用squid proxy允許內網(Not Allow Internet Access)用戶透過proxy啟動Windows 2008 VM

參考資料:

http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-only-forwards-GET-requests-to-cache-peer-td4277600.html (設定兩層式proxy要forward SSL需增加Line 31的設定)

http://squid-web-proxy-cache.1019090.n4.nabble.com/Forwarding-HTTP-and-HTTPS-Traffic-to-an-Upstream-Proxy-using-Cache-Peer-on-separate-ports-td1026947.html

架構:

Internet <—->OA Proxy <—->private proxy <———> windows VM (待補圖形架構圖)

環境:

CentOS 6.4 x86_64 with squid 3.1

squid設定:

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

acl ms_1 dstdomain activation.sls.microsoft.com
http_access allow ms_1
acl ms_2 dstdomain urs.microsoft.com
http_access allow ms_2
acl ms_3 url_regex -i ^http://go.microsoft.com/fwlink
http_access allow ms_3

#CentOS update mirror site
acl centos_mirror dstdomain mirror01.xxx.yyy.net
http_access allow centos_mirror

cache_peer 10.22.33.44 parent 8080 0 no-query no-digest originserver name=oaproxy login=PASSTHRU
cache_peer_access oaproxy allow ms_1
cache_peer_access oaproxy allow ms_2
cache_peer_access oaproxy allow ms_3
cache_peer_access oaproxy allow centos_mirror
cache_peer_access oaproxy deny all

visible_hostname squid

logformat squid %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %un %Sh/%<A %mt
logformat squidmime %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %un %Sh/%<A %mt [%>h] [%<h]
logformat common %>a %ui %un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st %Ss:%Sh
logformat combined %>a %ui %un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
access_log /var/log/squid/access.log combined

nonhierarchical_direct off

Brocade VDX6710 設定備忘

當Layer2 switch用的指令備忘

firmware upgrade

# mount usb stick
usb on

#download and upgrage firmware
firmware download usb directory nos4.0.0b

停用VCS (會reboot設備)

no vcs enable

把TenGigabit介面設定到vlan200

configure terminal
interface TenGigabitEthernet 0/49
no shutdown
switchport
switchport access vlan 200
top
interface TenGigabitEthernet 0/50
no shutdown
switchport
switchport access vlan 200
top
interface TenGigabitEthernet 0/51
no shutdown
switchport
switchport access vlan 200
top
interface TenGigabitEthernet 0/52
no shutdown
switchport
switchport access vlan 200
top
interface TenGigabitEthernet 0/53
no shutdown
switchport
switchport access vlan 200
top
interface TenGigabitEthernet 0/54
no shutdown
switchport
switchport access vlan 200
top

configure terminal
interface TenGigabitEthernet 0/49
mtu 9000
top

儲存設定

copy running-config startup-config

cacti 資料精確度由5mins改成1mins

環境:

CentOS 6.4 x86_64版本+EPEL repo

(必須是要全新安裝cacti才適用，舊的5mins資料無法直接升級到1min)

安裝cacti bin rpm

yum -y install cacti

使用下列指令建立及載入預設資料庫

mysqladmin -u root -p create cacti
mysql -u root -p cacti < /usr/share/doc/cacti-0.8.8b/cacti.sql

要在cacti資料庫執行下列作業

# ##待驗證##
# 參考:https://www.tolaris.com/2013/07/09/cacti-and-1-minute-polling/

TRUNCATE TABLE rra;
INSERT INTO `rra` (`id`, `hash`, `name`, `x_files_factor`, `steps`, `rows`, `timespan`) VALUES
(1, 'c21df5178e5c955013591239eb0afd46', 'Daily (5 Minute Average)', 0.5, 5, 25920, 86400),
(2, '0d9c0af8b8acdc7807943937b3208e29', 'Weekly (30 Minute Average)', 0.5, 30, 9000, 604800),
(3, '6fc2d038fb42950138b0ce3e9874cc60', 'Monthly (2 Hour Average)', 0.5, 120, 2250, 2592000),
(4, 'e36f3adb9f152adfa5dc50fd2b23337e', 'Yearly (1 Day Average)', 0.5, 1440, 797, 33053184),
(5, '283ea2bf1634d92ce081ec82a634f513', 'Hourly (1 Minute Average)', 0.5, 1, 10080, 14400);

DELETE FROM `data_template_data_rra` WHERE `rra_id`=5;
INSERT INTO `data_template_data_rra` SELECT DISTINCT `data_template_data_id`,5 FROM `data_template_data_rra`;

UPDATE `data_template_rrd` SET rrd_heartbeat =120;
UPDATE `data_template_data` SET rrd_step =60;

修改/etc/cron.d/cacti

*/1 * * * *    cacti   /usr/bin/php /usr/share/cacti/poller.php > /dev/null 2>&1

改為1mins後會增加系統負載，建議改用spine方式更新snmp

http://www.cacti.net/downloads/spine/

VMWare ESXi 5.0重新載入(reset)網卡driver

今天Qlogic QLE8152的10G CNA死當造成後端NFS storage連不到

緊急、臨時解法：

透過SSH登入ESXi主機後執行下列指令(若是ssh mgmt網段與要reset網卡在同一網段的話不可使用)

vmkload_mod -u qlge
vmkload_mod qlge

todo:後續需注意firmware/driver的更新

使用powercli批次更換VM datastore

參考資料: http://winblog.ch/2012/03/19/finding-and-updating-virtual-machines-with-outdated-tools-on-vsphere-using-powershell/

Connect-VIServer -Server 10.10.10.200 -User admin -Password 'testpass'
$VMs=get-vm
foreach ($vm in $VMs)
{
    $VMHost_Name = $vm.vmhost.name
    if ($VMHost_Name -eq "10.10.10.11")
    {
        Write-Host "Change datastore of" $VM.Name
        Move-VM -vm $VM.Name -datastore local_240
    }
}

CentOS 6.4 cacti 0.8.8.b-1無法顯示snmp_informant縮圖bug解法

環境：

CentOS 6.4 x86_64 + EPEL repo

cacti 0.8.8.b-1 (from EPEL)

snmp_informant cacti template (https://github.com/mrlesmithjr/cacti/)

問題說明：

cacti中使用縮圖方式無法顯示與snmp_informant template相關的圖檔

解決方式：

參考資料 (手動patch或是直接用yum升級cacti到0.8.8.b-3即可)

yum -y update cacti

使用powercli批次更新VM tools

今天把兩台ESXi 4.1升級到5.0，發現數十台VM的vm tool都過期需要更新，從vcenter一台一台點選更新實在不太有效率，所以就要靠powercli的幫忙，把更新的動作script化

google找到的參考資料(http://winblog.ch/2012/03/19/finding-and-updating-virtual-machines-with-outdated-tools-on-vsphere-using-powershell/)，除了有我要的powercli現成範例之外也說明了如何使用powercli的一些概念，對於初學powercli的人很有幫助

Connect-VIServer your-vi-server
$VMs = Get-VM
foreach ($vm in $VMs)
{
	$ToolsStatus = $vm.ExtensionData.Guest.ToolsStatus
	if ($ToolsStatus -eq "toolsOld")
	{
		Write-Host "Updating the tools of" $VM.Name
		$vm | Update-Tools -NoReboot -RunAsync 
	}
}

注意事項:

line1需要輸入vCenter帳號密碼

line6有稍做微調僅更新有安裝vm tools且過期的VM