After spending many hours trying to get the WEP encryption to work I have finally found out the problem using WEP keys.
The examples that are given in the application notes (AN404) are wrong with the description of the WEP key arrays.
The given example of keys is.
unsigned key0[13] = { 0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56 };
unsigned key1[13] = { 0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56 };
unsigned key2[13] = { 0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56 };
unsigned key3[13] = { 0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56 };
The declaration of these keys should be as chars as in
unsigned char key0[13] = { 0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56 };
unsigned char key1[13] = { 0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56 };
unsigned char key2[13] = { 0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56 };
unsigned char key3[13] = { 0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56,0x78,0x90,0x12,0x34,0x56 };
The reason for this is the compiler assumes that they are integers so when the call is made to functions
wifi_ioctl(NULL,WIFI_WEP_KEY0,key0,sizeof(key0)); // set 128bit (13 bytes) key
wifi_ioctl(NULL,WIFI_WEP_KEY1,key1,sizeof(key1)); // set 128bit (13 bytes) key
wifi_ioctl(NULL,WIFI_WEP_KEY2,key2,sizeof(key2)); // set 128bit (13 bytes) key
wifi_ioctl(NULL,WIFI_WEP_KEY3,key3,sizeof(key3)); // set 128bit (13 bytes) key
then the sizeof function will assume that they are integers and not chars hence the size of the key is wrong
If the correct declaration is made then the WEP encryption works.
cheers