Did you know you can...

Configure NFS Connector with Apache Tez in HDInsight       

This recipe describes how to configure Apache Tez with the Netapp NFS Connector. This guide is specific to HDInsight clusters in Microsoft Azure. The steps in this guide can be generalized to any Hadoop cluster running Tez.

This recipe is specific to HDInsight clusters in Microsoft Azure. The stepscan be generalized to any Hadoop cluster running Tez.

1. Configure the user “hive” and the group “hadoop” across all workers in your Hadoop cluster.

  sudo /netappnfs/usergroup.py -u hive -g hadoop -t /netappnfs

2. Copy the Netapp NFS Connector jar files to the fs.defaultFS as configured in Ambari.

  hdfs dfs -mkdir /{path to desired location}/aux-jars

  hdfs dfs -copyFromLocal /netappnfs/hadoop-nfs-2.7.1.jar /{path to desired location}/aux-jars

  hdfs dfs -copyFromLocal /netappnfs/hadoop-nfs-connector-2.0.0.jar /{path to desired location}/aux-jars

3. Add the new auxiliary jar location to the tez-site.conf file (reference).

4. Add the auxiliary jar path using Ambari:

5. Manually edit tez-site.conf, adding the following lines:

  <tez.aux.uris>

  /{path to desired location}/aux-jars

  </tez.aux.uris>

6.Restart the Tez, Hive, Oozie, etc. services

Did you know you can....

Use QoS to prioritize business critical applications?

This recipe describes how to use the QoS Floor feature to guarantee a Business Critical Application gets its required service of 40,000 IOPS regardless of interference coming from the Test and Dev Applications.

1. In System Manager, select the Business Critical Volume followed by Actions, Storage QoS

2. Set the Minimum Throughput to 40,000IOPS

Check the  performance dashboard to verify that the service level guarantee of 40,000 IOPS is met and latency is reduced.

Note that the test and dev workloads may see impacts meaning fewer IOPS and higher latencies.

For more information, visit the ONTAP 9 Documentation Center.

Hi, 

I am new to the ONTAP API and I am looking for a way to get space used by an aggregate in C#. How can I do that?
I know I need to pass a specific function to the NaElement object, but how do I search for that function? I am using API version 1.9

Thanks

Hey everyone we just got a new 2650 in and we are working on setting it up. We will have a 2 node cluster and on one node we are wanting to assign an all ssd flash aggregate to host our VMware environment on NFS and on the other node we will host NFS, iscsi, and cifs on sas drives with a SSD flash pool assigned to it.

Here comes our problem. When we initialize the 2650 it's taking 10 sas drives for node 1's root volume using ADP. All that space will be wasted because we have no intentions of using spinning aggregates on node 1. We really want to assign 4 sas drives and avoid eating 6 drives. What's the best way to go about this? I know there is a way to move root aggregates but we tried the 9.0 method and it doesn't seem to work correctly. My idea was turn off all shelves and leave only 8 drives left plugged in and reset everything with option 4. Theoretically it would take 4 drives for node 1 and 4 drives 4 node 2 and I would be where I want to be. Anyone see anything wrong with this? Should it be 95% full even after initial setup?

I called support and they told me they can't help since it's a new setup and techinically this is a new setup and we would have to get professional services even though in the guides on moving root volumes they say to work with support if you don't go with their built in setup

Any help would be greatly appreciated.

Thanks!

We are 100% NAS but recently added FC to our FAS8080 EX (cDOT 9.2) for legacy ESXi datastores (cannot be NFS).  Should we create the FC LUNs in the same SVM as our NAS volumes or do folks tend to have separate SVMs for SAN and NAS protocols?

I am seeing CPU activity 75% - 90% on average,  using Harvester I can see that the node aggr0 is unusually busy.  Autosupport reports this

bug (see url below) is present, which I suspect maybe the cause of the high CPU and aggr0 / root volume high activity.

https://mysupport.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=1061496

However I am not sure how to fix it ? Its my understanding access to the systemshell is limited to support ? 

so how can I get access to the file /var/etc/periodic.conf.local to carry out the work around?

https://docs.netapp.com/ontap-9/index.jsp?topic=%2Fcom.netapp.doc.dot-cm-vsmg%2FGUID-A60F4F83-0034-4DB8-838B-06E9D4BEF9A4.html&resultof=%22%73%79%73%74%65%6d%73%68%65%6c%6c%22%20%22%73%79%73%74%65%6d%73%68%65%6c%22%20

Problem node:

hncl1::*> node run -node hncl1-01 -command sysstat -d   
 CPU     NFS    CIFS    HTTP     Net   kB/s     HDD   kB/s     SSD   kB/s    Tape   kB/s  Cache
                                  in    out    read  write    read  write    read  write    age
 89%   15055     212       0   57821  20892  666163     21    2496      0       0      0    17s
 92%   15433     232       0  128528  17175  715690   1153    2505      0       0      0    17s
 86%   15850     520       0   49546  30543  546324     16    2647      0       0      0    28s
 82%   16405     254       0   21778  38242  462387      5    1889      0       0      0    14s
 89%   15776     232       0   82837  14417  700850     16    2069      0       0      0    14s
 89%   15344     234       0   45265  12588  661101 153785    6578  10992       0      0    14s
 87%   13749     533       0   20167  13852  650736 132215    1648   5481       0      0    14s

Good Node as example:

hncl1::*> node run -node hncl1-02 -command sysstat -d
CPU NFS CIFS HTTP Net kB/s HDD kB/s SSD kB/s Tape kB/s Cache
in out read write read write read write age
35% 2540 3 0 13430 14434 130278 16 3182 0 0 0 22s
37% 3007 1 0 15732 18484 43973 0 4699 0 0 0 24s
32% 3123 9 0 13962 14882 16851 21 4424 0 0 0 24s
31% 3213 2 0 13745 14351 12757 0 4600 0 0 0 24s
31% 2933 6 0 13607 14751 15683 16 4115 0 0 0 19s
32% 2839 1 0 27327 15809 27155 5 3539 0 0 0 19s
39% 3673 3 0 19517 17364 39400 65240 10589 14541 0 0 23s
35% 3361 5 0 16045 19323 17016 6499 4811 909 0 0 23s
37% 2846 3 0 20705 19875 19470 21 5300 0 0 0 23s

Having the HIGH CPU also means I cannot perform a NDU to get to a higher level where the bug is fixed.

Once I can get my CPU under control I get performance  the NDU.

I hope someone can please provide some guidance. 

Hi,

With regards to initialization of disks, does selecting Option (4) Clean configuration and initialize all disks encapsulate going into (5) Maintenance mode boot and doing "aggr destroy aggr0" -> storage release disks -> disk remove_ownership all?

Thanks.

Hi All,

I'm really confused about what's happening.

On any RH 6.x I'm not able to mount with any NFS version any qtree exported in my volume. Same exported qtrees work fine with either RH 5.x and RH 7.x, no matter if I choose version 3 or 4.

Error is always access denied.

Same 6.x servers works fine in 7-Mode environment

Path to the qtree is correcty enabled at read level for all the servers.

Any idea?

Many Thanks

Stefano

This guide addresses the NetApp® ONTAP® 9 data management software solution. TheONTAP solution unifies the storage infrastructure, often referred to as the NetApp DataFabric. In addition, ONTAP allows an organization to combine traditional datacenters and new architectures into a unified data fabric. The organization can then move,store, and manage data across flash, disk, and cloud resources in a unified manner. Theguidance and configuration settings in this document are designed to help organizationsmeet prescribed security objectives for information system confidentiality, integrity, andavailability.

for more info, please check here

Hi,

I am trying to configure 2 nodes (with 3 stacks of disks) that we just added to our lab environment. These nodes were decommissioned from the production environment. The disks in 2 stacks are displaying as broken with one stack showing amber light on all but 3 disks.

Please can someone direct me as to how to solve this problem(s) as I am new to Netapp Storage?

xxxxxxxx::*> storage disk show -container-type broken
                     Usable           Disk    Container   Container
Disk                   Size Shelf Bay Type    Type        Name      Owner
---------------- ---------- ----- --- ------- ----------- --------- --------
4.10.3              836.9GB    10   3 SAS     broken      -         yyyyy
4.10.4              836.9GB    10   4 SAS     broken      -         yyyyy
4.10.5              836.9GB    10   5 SAS     broken      -         yyyyy
4.10.6              836.9GB    10   6 SAS     broken      -         yyyyy

.

.

5.50.10              3.63TB    50  10 FSAS    broken      -        yyyyy
5.50.11              3.63TB    50  11 FSAS    broken      -        yyyyy
5.50.12              3.63TB    50  12 FSAS    broken      -        yyyyy

.

.

.

Thanks.

Hello,

we have older snapshot, which are created during an ndmp backup task. In the OnCommand System Manager they have the Status "Busy". We can't delete them.

On the SSH Console we see that there are active NDMP sessions/tasks:

filer::> system node run -node filer-01 -command backup status

ID State Type Device Start Date Level Path
-- ----------- ---- ------ ------------ ----- ---------------
0 ACTIVE NDMP ndmp Sep 20 09:32 0 /svm_*
3 ACTIVE NDMP ndmp Sep 20 08:50 0 /svm_*
6 ACTIVE NDMP ndmp Sep 20 09:01 0 /svm_*
9 ACTIVE NDMP ndmp Sep 20 09:35 0 /svm_*
10 ACTIVE NDMP ndmp Sep 20 08:55 0 /svm_*

We can't terminate them, because the state is ACTIVE. Also a the command "vserver services ndmp kill-all -vserver *" didn't solve the problem.

Does anybody havae an idea how to delete these snapshots?

Regards,

Christian

Hi there,

I want/need to leverage either the NetApp shell or the NetApp powershell modules in order to apply NTFS permissions at a (way) higher speed. I have a lot of very large shares (millions of files each) and need to revisit their permissions. Doing it in Windows, even on a server on the same network as the storage, takes forever.

The catch: I need to set up the permisisons for a specific account twice. So I have two allow entries for the same account. To make it simpler to describe, let's say that DOMAIN\GroupA needs Modify rights on Subfolders and Files and Read on This folder only.

I know how to apply permissions with both the PS module and with the shell, I know I need a policy etc. It all works when I don't use the same account twice. So I'm only gonna post the two lines of codes that should do what I need).

In powershell, the two lines for this specific setting would be ($VolName is the share and $VServer the netapp appliance):

Add-NcFileDirectorySecurityNtfsDacl -Account "DOMAIN\GroupA" -AccessType allow -Rights modify -NtfsSd $VolName -VserverContext $Vserver -ApplyTo sub_folders,files
Add-NcFileDirectorySecurityNtfsDacl -Account "DOMAIN\GroupA" -AccessType allow -Rights read -NtfsSd $VolName -VserverContext $Vserver -ApplyTo this_folder

In the shell (also works with a single entry) the two lines of code would be:

vserver security file-directory ntfs dacl add -ntfs-sd MYSHARE -vserver VSERVERNAME -access-type allow -account "domain\GroupA" -rights modify -apply-to sub-folders,files
vserver security file-directory ntfs dacl add -ntfs-sd MYSHARE -vserver VSERVERNAME -access-type allow -account "domain\GroupA" -rights read -apply-to this-folder

Here's the error from PS:

Shell error:

I also saw somebody else with the same error here:
https://community.netapp.com/t5/Microsoft-Cloud-and-Virtualization-Discussions/NetApp-PowerShell-Toolkit-4-5-released/td-p/135904

Any help is greatly appreciated!
Thanks.

On our source cluster, we have a volume that does not have a snapshot policy associated with it, as we are manually creating the snapshots from a command line (they are named hourly.20171218.09 as an example).

We created a SnapVault policy with our other cluster, but none of the manually created snapshots are synced and the "Policy Details" section of OnCOmmand manager shows the label as "hourly" but it says "Source does not have any schedules with this label".

Is there a way to SnapVault snapshots that are manually created or is the only way to have the snapshots created via a snapshot policy?

Good day all,

Have not had any luck in finding ways to delete snapshots from a FlexGroup and how do you disable the default snapshot schedule just for the FlexGroup and it's constituents.  Any suggesstions would be most helpful.

Thank you

I ran fsecurity apply on a folder on my filer (NetApp Release 8.2.4P4 7-Mode) to apply new ACLs to the folder and all subfolders and files in it. Here is the contents of the security.conf file I applied:

cb56f6f4
1,0,"/vol/vol1/topics/DL/",0,"D:_(A;CIOI;0x1f01ff;;;DE\filer_users)" (The underscore is actually not there)

fsecurity finished sucessfully and looking at the ACLs from windows looked good.

We also mount this share on our Linux computers (Ubuntu 16.04, NFS vers. 4). To my surprise the Linux permissions were all removed and looked like this:

d--------- 13 root    nobody 4096 Dec  4 16:30 docs
----------  1 fred    nobody 9728 Dec 20 09:48 Thumbs.db

Also, the group was changed to "nobody". This is causing problems for the users. If you try to write a file, you get a "read only" warning. Writing is still possible, since the ACL allows writing, but you have to explicitly tell the programs to ignore the "read only" warning.

Since chmod and chgrp commands are ignored, how can I set the permission mode to something like "rwxrwx---", which should be the default? What have I done wrong with the fsecurity apply run?

Thank you,

Andreas

Hi,

My client code (C++) using ONTAP API (management SDK 5.5) worked fine with the cluster (running 8.3) before.

But after the cluster is upgraded to 9.1P9, the client cannot connect to the cluster with the following error:

Can't connect to host (err=5)

I also tried upgrading my management SDK to 9.3, but it failed with the same error.

Any idea on what is going wrong, and how should I fix this? Is it because the Ontap server is not running properly? If so, how should I diagnose in this case?

Thank you very much in advance.

PS: I am also attaching the tcpdump log when the client was trying to connect, just in case it might help:

15:17:37.297091 IP (tos 0x0, ttl 64, id 56756, offset 0, flags [DF], proto TCP (6), length 60)

    client_hostname.38530 > netapp_cluster_hostname.https: Flags [S], cksum 0x6b7d (incorrect -> 0x8c2e), seq 4280977165, win 26880, options [mss 8960,sackOK,TS val 1359436473 ecr 0,nop,wscale 7], length 0

15:17:37.297390 IP (tos 0x0, ttl 63, id 45324, offset 0, flags [DF], proto TCP (6), length 60)

    netapp_cluster_hostname.https > client_hostname.38530: Flags [S.], cksum 0x40cb (correct), seq 1555209334, ack 4280977166, win 65535, options [mss 8960,nop,wscale 6,sackOK,TS val 2784367924 ecr 1359436473], length 0

15:17:37.297411 IP (tos 0x0, ttl 64, id 56757, offset 0, flags [DF], proto TCP (6), length 52)

    client_hostname.38530 > netapp_cluster_hostname.https: Flags [.], cksum 0x6b75 (incorrect -> 0x8c10), seq 1, ack 1, win 210, options [nop,nop,TS val 1359436473 ecr 2784367924], length 0

15:17:37.300312 IP (tos 0x0, ttl 64, id 56758, offset 0, flags [DF], proto TCP (6), length 347)

    client_hostname.38530 > netapp_cluster_hostname.https: Flags [P.], cksum 0x6c9c (incorrect -> 0xa5f5), seq 1:296, ack 1, win 210, options [nop,nop,TS val 1359436476 ecr 2784367924], length 295

15:17:37.500489 IP (tos 0x0, ttl 64, id 56759, offset 0, flags [DF], proto TCP (6), length 347)

    client_hostname.38530 > netapp_cluster_hostname.https: Flags [P.], cksum 0x6c9c (incorrect -> 0xa52c), seq 1:296, ack 1, win 210, options [nop,nop,TS val 1359436677 ecr 2784367924], length 295

15:17:37.500741 IP (tos 0x0, ttl 63, id 45326, offset 0, flags [DF], proto TCP (6), length 52)

    netapp_cluster_hostname.https > client_hostname.38530: Flags [.], cksum 0x7ffe (correct), seq 1575, ack 296, win 1024, options [nop,nop,TS val 2784368127 ecr 1359436677], length 0

15:18:56.676793 IP (tos 0x0, ttl 63, id 46685, offset 0, flags [DF], proto TCP (6), length 52)

    netapp_cluster_hostname.https > client_hostname.38530: Flags [R.], cksum 0x4ab1 (correct), seq 1575, ack 296, win 1024, options [nop,nop,TS val 2784447303 ecr 1359436677], length 0

15:18:56.676829 IP (tos 0x0, ttl 64, id 56760, offset 0, flags [DF], proto TCP (6), length 52)

    client_hostname.38530 > netapp_cluster_hostname.https: Flags [.], cksum 0x6b75 (incorrect -> 0x54d4), seq 296, ack 1, win 210, options [nop,nop,TS val 1359515853 ecr 2784367924], length 0

15:18:56.677093 IP (tos 0x0, ttl 63, id 46686, offset 0, flags [DF], proto TCP (6), length 40)

    netapp_cluster_hostname.https > client_hostname.38530: Flags [R], cksum 0xb32a (correct), seq 1555209335, win 0, length 0

I have a flexpod setup and we just installed a new exchange server as a VMware virtual machine on an NFS datastore. I am trying to figure out the best way to backup the data. I have used Snapmanger for sql in the past, but that was on iscsi LUNs. Can snapmanager work on NFS storage too? I just want to find the most efficient way to backup data on the server. I can use commvault if neccessary but snapshots are so much simpler. Any information/direction would be appreciated.

Thanks!

Hello everyone,

There is a hybrid aggregate with 800GB SSD.

Can we add 3.8TB SSD to this aggregate to increase the cache size ?

Is there any experience about that?

Regards.

We are looking at various options for upgrading the NetApp environment in our remote offices from 7-mode 2220s to either cdot 2620s, commodity hardware with cdot OnTAP Select, or possibly HCI running cdot OnTAP Select. Does anyone know if you can perform 7-mode transitions to OnTAP Select running on HCI hardware?